[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:56:46 dist-test-slave-9cqd krb5kdc[23038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:56:46 dist-test-slave-9cqd krb5kdc[23038](info): set up 2 sockets
Sep 28 15:56:46 dist-test-slave-9cqd krb5kdc[23038](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 15:56:47.106395 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.006s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:56:47 dist-test-slave-9cqd krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:56:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:56:47.276748 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:37589
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:37589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:47.629730 23051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:47.630075 23051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:47.630240 23051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:47.637308 23051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:56:47.637434 23051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:47.637534 23051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:47.637636 23051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:56:47.637722 23051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:56:47.649878 23051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:37589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:37589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:47.652590 23051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:47.654392 23051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:47.673894 23056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:47.673844 23059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:47.673693 23057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:47.674733 23051 server_base.cc:1034] running on GCE node
I20260928 15:56:47.675685 23051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:47.677767 23051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:47.678997 23051 hybrid_clock.cc:648] HybridClock initialized: now 1790611007678917 us; error 76 us; skew 500 ppm
I20260928 15:56:47.679435 23051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:47 dist-test-slave-9cqd krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611007, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:47.686270 23051 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:56:47.688472 23051 webserver.cc:460] Webserver started at http://127.22.125.254:36027/ using document root <none> and password file <none>
I20260928 15:56:47.689488 23051 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:47.689639 23051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:47.690151 23051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:47.694049 23051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "821f5efbb133451898090e811be9b2c6"
format_stamp: "Formatted at 2026-09-28 15:56:47 on dist-test-slave-9cqd"
server_key: "994af8f68b09785468aaa6f60b202a34"
server_key_iv: "c1796ebf0d5a1ae64ce67640e78c2ae4"
server_key_version: "encryptionkey@0"
I20260928 15:56:47.695099 23051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "821f5efbb133451898090e811be9b2c6"
format_stamp: "Formatted at 2026-09-28 15:56:47 on dist-test-slave-9cqd"
server_key: "994af8f68b09785468aaa6f60b202a34"
server_key_iv: "c1796ebf0d5a1ae64ce67640e78c2ae4"
server_key_version: "encryptionkey@0"
I20260928 15:56:47.703164 23051 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260928 15:56:47.708449 23066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:47.710268 23051 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260928 15:56:47.710459 23051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "821f5efbb133451898090e811be9b2c6"
format_stamp: "Formatted at 2026-09-28 15:56:47 on dist-test-slave-9cqd"
server_key: "994af8f68b09785468aaa6f60b202a34"
server_key_iv: "c1796ebf0d5a1ae64ce67640e78c2ae4"
server_key_version: "encryptionkey@0"
I20260928 15:56:47.710628 23051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:47.739216 23051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:47.743659 23051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:47.743894 23051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:47.765487 23117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:37589 every 8 connection(s)
I20260928 15:56:47.765496 23051 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:37589
I20260928 15:56:47.768196 23051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:56:47.772782 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23051
I20260928 15:56:47.773226 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:56:47.775357 23118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:47.776417 23031 external_mini_cluster.cc:1428] Setting key b360d2dca123527e42808cdc210a001e
I20260928 15:56:47.791846 23118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6: Bootstrap starting.
I20260928 15:56:47.797103 23118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:47.799206 23118 log.cc:824] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:47.803064 23118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6: No bootstrap required, opened a new log
I20260928 15:56:47.812475 23118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821f5efbb133451898090e811be9b2c6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37589 } }
I20260928 15:56:47.813028 23118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 28 15:56:47 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:47.813171 23118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821f5efbb133451898090e811be9b2c6, State: Initialized, Role: FOLLOWER
I20260928 15:56:47.814114 23118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821f5efbb133451898090e811be9b2c6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37589 } }
I20260928 15:56:47.814477 23118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:56:47.814752 23118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:56:47.814975 23118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:47.818636 23118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821f5efbb133451898090e811be9b2c6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37589 } }
I20260928 15:56:47.821015 23118 leader_election.cc:302] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 821f5efbb133451898090e811be9b2c6; no voters:
I20260928 15:56:47.821759 23118 leader_election.cc:288] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:56:47.821856 23123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:56:47.823415 23123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [term 1 LEADER]: Becoming Leader. State: Replica: 821f5efbb133451898090e811be9b2c6, State: Running, Role: LEADER
I20260928 15:56:47.824312 23123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821f5efbb133451898090e811be9b2c6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37589 } }
I20260928 15:56:47.825119 23118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:56:47.825278 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:47.783563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.22.125.254:37589)
0928 15:56:47.784600 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:47.784625 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:47.788685 (+ 4060us) server_negotiation.cc:407] Connection header received
0928 15:56:47.798783 (+ 10098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:47.798811 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:47.799566 (+ 755us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:47.800158 (+ 592us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:47.803276 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:47.805878 (+ 2602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:47.807450 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:47.808657 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:47.808893 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:47.814423 (+ 5530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:47.814501 (+ 78us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:47.814543 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:47.814608 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:47.818686 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:47.819540 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:47.819552 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:47.819560 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:47.819771 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:47.820233 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:47.820236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:47.820238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:47.821619 (+ 1381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:47.822074 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:47.823149 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0928 15:56:47.823663 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":356,"threads_started":1}
I20260928 15:56:47.835155 23124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 821f5efbb133451898090e811be9b2c6. Latest consensus state: current_term: 1 leader_uuid: "821f5efbb133451898090e811be9b2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821f5efbb133451898090e811be9b2c6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37589 } } }
I20260928 15:56:47.835664 23124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [sys.catalog]: This master's current role is: LEADER
I20260928 15:56:47.835242 23125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "821f5efbb133451898090e811be9b2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821f5efbb133451898090e811be9b2c6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37589 } } }
I20260928 15:56:47.837256 23125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6 [sys.catalog]: This master's current role is: LEADER
W20260928 15:56:47.841265 23138 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 15:56:47.841459 23138 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 15:56:47.841610 23139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:56:47.844430 23139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:56:47.857492 23139 catalog_manager.cc:1347] Generated new cluster ID: d14bd386354147408827f1b9f8ed37ef
I20260928 15:56:47.857620 23139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:56:47.866734 23139 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:56:47.868279 23139 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:56:47.879949 23139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 821f5efbb133451898090e811be9b2c6: Generated new TSK 0
I20260928 15:56:47.880923 23139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:56:47.924309 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37589
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:48.251694 23144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:48.252010 23144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:48.252223 23144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:48.259680 23144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:48.259812 23144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:48.259999 23144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:56:48.269771 23144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:48.271032 23144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:48.272446 23144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:48.285784 23150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:48.285908 23149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:48.285935 23152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:48.286700 23144 server_base.cc:1034] running on GCE node
I20260928 15:56:48.287366 23144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:48.288362 23144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:48.289588 23144 hybrid_clock.cc:648] HybridClock initialized: now 1790611008289543 us; error 35 us; skew 500 ppm
I20260928 15:56:48.289903 23144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:48 dist-test-slave-9cqd krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:48.292984 23144 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:56:48.294337 23144 webserver.cc:460] Webserver started at http://127.22.125.193:34619/ using document root <none> and password file <none>
I20260928 15:56:48.295024 23144 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:48.295178 23144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:48.295501 23144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:48.298393 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "a74bc7f293004197968d11d07c65f6cd"
format_stamp: "Formatted at 2026-09-28 15:56:48 on dist-test-slave-9cqd"
server_key: "b82fb8ff19033a88f4f6b680fc4e5125"
server_key_iv: "d81f1c4dbc02471611296b41345c6061"
server_key_version: "encryptionkey@0"
I20260928 15:56:48.299172 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "a74bc7f293004197968d11d07c65f6cd"
format_stamp: "Formatted at 2026-09-28 15:56:48 on dist-test-slave-9cqd"
server_key: "b82fb8ff19033a88f4f6b680fc4e5125"
server_key_iv: "d81f1c4dbc02471611296b41345c6061"
server_key_version: "encryptionkey@0"
I20260928 15:56:48.304174 23144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260928 15:56:48.307399 23159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:48.308692 23144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:56:48.308864 23144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "a74bc7f293004197968d11d07c65f6cd"
format_stamp: "Formatted at 2026-09-28 15:56:48 on dist-test-slave-9cqd"
server_key: "b82fb8ff19033a88f4f6b680fc4e5125"
server_key_iv: "d81f1c4dbc02471611296b41345c6061"
server_key_version: "encryptionkey@0"
I20260928 15:56:48.309057 23144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:48.331954 23144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:48.336009 23144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:48.336354 23144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:48.337354 23144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:56:48.338781 23144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:48.338892 23144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:48.339052 23144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:48.339126 23144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:48.372566 23144 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:39243
I20260928 15:56:48.372656 23271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:39243 every 8 connection(s)
I20260928 15:56:48.373893 23144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:56:48.381771 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23144
I20260928 15:56:48.382019 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:56:48.382762 23031 external_mini_cluster.cc:1428] Setting key 920592d5332910a2dedc9caad6647b0f
Sep 28 15:56:48 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:48.396863 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:48.377970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:38405 (local address 127.22.125.254:37589)
0928 15:56:48.378436 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:48.378443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:48.379453 (+ 1010us) server_negotiation.cc:407] Connection header received
0928 15:56:48.381392 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:48.381400 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:48.381495 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:48.381628 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:48.383266 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.384230 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.386059 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.386837 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.386995 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:48.390924 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:48.390929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:48.390932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:48.390964 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:48.393553 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:48.394268 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:48.394275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:48.394279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:48.394404 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:48.394972 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:48.394976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:48.394977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:48.395235 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:48.395344 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:48.396438 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0928 15:56:48.396614 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":339,"thread_start_us":222,"threads_started":1}
I20260928 15:56:48.398510 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:48.378533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37589 (local address 127.22.125.193:38405)
0928 15:56:48.379302 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:48.379340 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:48.381130 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:48.381817 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:48.381825 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:48.382467 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:48.383094 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:48.383114 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.384374 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.384379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:48.385904 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:48.385916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.387041 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.387046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:48.387384 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:48.388146 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:48.388168 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:48.390703 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:48.393696 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:48.393704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:48.393721 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:48.394144 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:48.394559 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:48.394566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:48.394573 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:48.394836 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:48.395384 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:48.395396 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:48.395890 (+ 494us) client_negotiation.cc:769] Sending connection context
0928 15:56:48.396378 (+ 488us) client_negotiation.cc:240] Negotiation successful
0928 15:56:48.396974 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":265,"threads_started":1}
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:56:48.400128 23272 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37589
I20260928 15:56:48.400519 23272 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:48.401347 23272 heartbeater.cc:511] Master 127.22.125.254:37589 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:56:48.406788 23083 ts_manager.cc:194] Registered new tserver with Master: a74bc7f293004197968d11d07c65f6cd (127.22.125.193:39243)
I20260928 15:56:48.410286 23083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:38405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:56:48.421271 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37589
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:48.746836 23278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:48.747120 23278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:48.747219 23278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:48.754370 23278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:48.754491 23278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:48.754608 23278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:56:48.764149 23278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:48.765805 23278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:48.767571 23278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:48.782346 23283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:48.782354 23284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:48.783159 23286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:48.783648 23278 server_base.cc:1034] running on GCE node
I20260928 15:56:48.784291 23278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:48.785315 23278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:48.786547 23278 hybrid_clock.cc:648] HybridClock initialized: now 1790611008786497 us; error 28 us; skew 500 ppm
I20260928 15:56:48.786944 23278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:48 dist-test-slave-9cqd krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:48.790690 23278 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:56:48.793081 23278 webserver.cc:460] Webserver started at http://127.22.125.194:41919/ using document root <none> and password file <none>
I20260928 15:56:48.793757 23278 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:48.793913 23278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:48.794219 23278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:48.797175 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "6c1d9ac9eb1b4f66b0aeba78915b471b"
format_stamp: "Formatted at 2026-09-28 15:56:48 on dist-test-slave-9cqd"
server_key: "bf727a686a438e1528e0d9d53586d3e2"
server_key_iv: "17b4765a4dd163664abe058cd1292cdd"
server_key_version: "encryptionkey@0"
I20260928 15:56:48.797971 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "6c1d9ac9eb1b4f66b0aeba78915b471b"
format_stamp: "Formatted at 2026-09-28 15:56:48 on dist-test-slave-9cqd"
server_key: "bf727a686a438e1528e0d9d53586d3e2"
server_key_iv: "17b4765a4dd163664abe058cd1292cdd"
server_key_version: "encryptionkey@0"
I20260928 15:56:48.803081 23278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260928 15:56:48.806536 23293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:48.807817 23278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 15:56:48.807992 23278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "6c1d9ac9eb1b4f66b0aeba78915b471b"
format_stamp: "Formatted at 2026-09-28 15:56:48 on dist-test-slave-9cqd"
server_key: "bf727a686a438e1528e0d9d53586d3e2"
server_key_iv: "17b4765a4dd163664abe058cd1292cdd"
server_key_version: "encryptionkey@0"
I20260928 15:56:48.808241 23278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:48.835088 23278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:48.839030 23278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:48.839327 23278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:48.840341 23278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:56:48.841676 23278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:48.841782 23278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:48.841933 23278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:48.842006 23278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:48.875357 23278 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:46571
I20260928 15:56:48.875449 23405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:46571 every 8 connection(s)
I20260928 15:56:48.876688 23278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:56:48.885617 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23278
I20260928 15:56:48.885811 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:56:48.886324 23031 external_mini_cluster.cc:1428] Setting key 955850424069a43f02caf3ff1facf9c8
Sep 28 15:56:48 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611008, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:48.896193 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:48.879206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:42741 (local address 127.22.125.254:37589)
0928 15:56:48.879391 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:48.879396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:48.880719 (+ 1323us) server_negotiation.cc:407] Connection header received
0928 15:56:48.882504 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:48.882508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:48.882588 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:48.882677 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:48.884131 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.884841 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.886534 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.886938 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.887074 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:48.890553 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:48.890558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:48.890562 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:48.890594 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:48.893206 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:48.893920 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:48.893925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:48.893931 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:48.894065 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:48.894496 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:48.894502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:48.894510 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:48.894760 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:48.894919 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:48.895587 (+ 668us) server_negotiation.cc:299] Negotiation successful
0928 15:56:48.895823 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 15:56:48.897157 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:48.879698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37589 (local address 127.22.125.194:42741)
0928 15:56:48.880559 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:48.880598 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:48.882256 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:48.882881 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:48.882889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:48.883388 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:48.883942 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:48.883960 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.884971 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.884975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:48.886354 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:48.886390 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:48.887158 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:48.887164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:48.887439 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:48.888236 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:48.888257 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:48.890337 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:48.893349 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:48.893358 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:48.893375 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:48.893792 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:48.894177 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:48.894182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:48.894186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:48.894371 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:48.894889 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:48.894896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:48.895240 (+ 344us) client_negotiation.cc:769] Sending connection context
0928 15:56:48.895494 (+ 254us) client_negotiation.cc:240] Negotiation successful
0928 15:56:48.896024 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":340,"threads_started":1}
I20260928 15:56:48.898689 23406 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37589
I20260928 15:56:48.899092 23406 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:48.900012 23406 heartbeater.cc:511] Master 127.22.125.254:37589 requested a full tablet report, sending...
I20260928 15:56:48.901618 23083 ts_manager.cc:194] Registered new tserver with Master: 6c1d9ac9eb1b4f66b0aeba78915b471b (127.22.125.194:46571)
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 15:56:48.902454 23083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:42741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:56:48.924566 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37589
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:49.245875 23411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:49.246132 23411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:49.246279 23411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:49.253279 23411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:49.253396 23411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:49.253551 23411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:56:49.262915 23411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:49.264199 23411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:49.265498 23411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:49.278518 23416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:49.278620 23419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:49.278548 23417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:49.279479 23411 server_base.cc:1034] running on GCE node
I20260928 15:56:49.280133 23411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:49.281147 23411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:49.282428 23411 hybrid_clock.cc:648] HybridClock initialized: now 1790611009282353 us; error 49 us; skew 500 ppm
I20260928 15:56:49.282773 23411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:49.285876 23411 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:56:49.287364 23411 webserver.cc:460] Webserver started at http://127.22.125.195:35321/ using document root <none> and password file <none>
I20260928 15:56:49.288034 23411 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:49.288240 23411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:49.288601 23411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:49.291455 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "66c8d482f50f4ddba14793b9bf759806"
format_stamp: "Formatted at 2026-09-28 15:56:49 on dist-test-slave-9cqd"
server_key: "8ff9445d8aec585bfb08372fbb0e4bfd"
server_key_iv: "dce924e1448698e7d7929626b2726526"
server_key_version: "encryptionkey@0"
I20260928 15:56:49.292220 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "66c8d482f50f4ddba14793b9bf759806"
format_stamp: "Formatted at 2026-09-28 15:56:49 on dist-test-slave-9cqd"
server_key: "8ff9445d8aec585bfb08372fbb0e4bfd"
server_key_iv: "dce924e1448698e7d7929626b2726526"
server_key_version: "encryptionkey@0"
I20260928 15:56:49.297137 23411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 15:56:49.300328 23426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:49.301601 23411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 15:56:49.301764 23411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "66c8d482f50f4ddba14793b9bf759806"
format_stamp: "Formatted at 2026-09-28 15:56:49 on dist-test-slave-9cqd"
server_key: "8ff9445d8aec585bfb08372fbb0e4bfd"
server_key_iv: "dce924e1448698e7d7929626b2726526"
server_key_version: "encryptionkey@0"
I20260928 15:56:49.301956 23411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:49.321473 23411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:49.325517 23411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:49.325762 23411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:49.326707 23411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:56:49.327919 23411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:49.327993 23411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:49.328142 23411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:49.328192 23411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:49.360013 23411 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:41139
I20260928 15:56:49.360095 23538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:41139 every 8 connection(s)
I20260928 15:56:49.361326 23411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:56:49.370232 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23411
I20260928 15:56:49.370424 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:56:49.370851 23031 external_mini_cluster.cc:1428] Setting key a5d36e77a0c67271d1221d05912461d7
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:49.382009 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:49.364066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:50989 (local address 127.22.125.254:37589)
0928 15:56:49.364242 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:49.364246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:49.365366 (+ 1120us) server_negotiation.cc:407] Connection header received
0928 15:56:49.367210 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:49.367216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:49.367324 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:49.367445 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:49.368988 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.369751 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.371467 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.371957 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.372110 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:49.376263 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:49.376270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:49.376274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:49.376315 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:49.379155 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.379829 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.379835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.379838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.379978 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.380431 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.380435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.380437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.380686 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:49.380840 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:49.381455 (+ 615us) server_negotiation.cc:299] Negotiation successful
0928 15:56:49.381668 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 15:56:49.382879 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:49.364503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37589 (local address 127.22.125.195:50989)
0928 15:56:49.365191 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:49.365238 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:49.366943 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:49.367671 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:49.367679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:49.368271 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:49.368816 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:49.368832 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.369895 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.369907 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:49.371304 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:49.371315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.372214 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.372224 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:49.372537 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:49.373265 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:49.373303 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:49.375997 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:49.379300 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:49.379308 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:49.379328 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:49.379700 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:49.380115 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:49.380120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:49.380123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:49.380296 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:49.380812 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:49.380820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:49.381115 (+ 295us) client_negotiation.cc:769] Sending connection context
0928 15:56:49.381362 (+ 247us) client_negotiation.cc:240] Negotiation successful
0928 15:56:49.381832 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":271,"threads_started":1}
I20260928 15:56:49.384205 23539 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37589
I20260928 15:56:49.384462 23539 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:49.385109 23539 heartbeater.cc:511] Master 127.22.125.254:37589 requested a full tablet report, sending...
I20260928 15:56:49.386581 23083 ts_manager.cc:194] Registered new tserver with Master: 66c8d482f50f4ddba14793b9bf759806 (127.22.125.195:41139)
I20260928 15:56:49.387565 23083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:50989
I20260928 15:56:49.387676 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260928 15:56:49.413851 23272 heartbeater.cc:503] Master 127.22.125.254:37589 was elected leader, sending a full tablet report...
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:56:49.433043 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:49.418283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.22.125.193:39243)
0928 15:56:49.418903 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:49.418910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:49.418939 (+ 29us) server_negotiation.cc:407] Connection header received
0928 15:56:49.419003 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:49.419009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:49.419212 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:49.419352 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:49.420776 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.421770 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.422954 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.423495 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.423590 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:49.427492 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:49.427503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:49.427517 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:49.427551 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:49.430429 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.431098 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.431104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.431106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.431232 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.431637 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.431643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.431649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.431994 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:49.432287 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:49.432524 (+ 237us) server_negotiation.cc:299] Negotiation successful
0928 15:56:49.432716 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":343,"threads_started":1}
W20260928 15:56:49.434381 23187 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54658
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:56:49.477475 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:49.463812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.22.125.193:39243)
0928 15:56:49.464078 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:49.464085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:49.464348 (+ 263us) server_negotiation.cc:407] Connection header received
0928 15:56:49.464572 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:49.464576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:49.464673 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:49.464788 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:49.466116 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.466825 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.468032 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.468531 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.468630 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:49.472657 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:49.472662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:49.472665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:49.472700 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:49.475312 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.475868 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.475871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.475874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.475995 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.476381 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.476384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.476386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.476630 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:49.476831 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:49.477047 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 15:56:49.477245 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 15:56:49.480397 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23144
I20260928 15:56:49.494331 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:49.482509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.22.125.193:39243)
0928 15:56:49.482702 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:49.482711 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:49.483104 (+ 393us) server_negotiation.cc:407] Connection header received
0928 15:56:49.483191 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:49.483195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:49.483284 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:49.483361 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:49.484534 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.485220 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.486426 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.486861 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.486960 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:49.489357 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:49.489361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:49.489364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:49.489394 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:49.492013 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.492644 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.492648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.492650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.492779 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.493076 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.493079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.493081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.493363 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:49.493536 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:49.493766 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 15:56:49.493918 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 15:56:49.495062 23267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:49.579604 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23144
I20260928 15:56:49.595924 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23278
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:56:49.611544 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:49.596869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.22.125.194:46571)
0928 15:56:49.597508 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:49.597515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:49.597540 (+ 25us) server_negotiation.cc:407] Connection header received
0928 15:56:49.597600 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:49.597605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:49.597819 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:49.597951 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:49.599136 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.600228 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.601417 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.601975 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.602073 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:49.606043 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:49.606054 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:49.606068 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:49.606099 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:49.609077 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.609702 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.609709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.609711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.609823 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.610162 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.610165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.610167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.610496 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:49.610721 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:49.610964 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 15:56:49.611207 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":352,"threads_started":1}
I20260928 15:56:49.612984 23401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:49.696091 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23278
I20260928 15:56:49.712458 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23411
Sep 28 15:56:49 dist-test-slave-9cqd krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:56:49.728441 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:49.713634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.22.125.195:41139)
0928 15:56:49.714284 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:49.714292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:49.714320 (+ 28us) server_negotiation.cc:407] Connection header received
0928 15:56:49.714383 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:49.714389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:49.714655 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:49.714829 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:49.716007 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.717099 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.718288 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:49.718831 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:49.718931 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:49.722811 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:49.722821 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:49.722836 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:49.722870 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:49.725887 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.726569 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.726575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.726578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.726695 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:49.726991 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:49.726994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:49.726996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:49.727306 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:49.727524 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:49.727736 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 15:56:49.727929 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":307,"threads_started":1}
I20260928 15:56:49.729681 23534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:49.812862 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23411
I20260928 15:56:49.828438 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23051
I20260928 15:56:49.829591 23113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:49.933590 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23051
2026-09-28T15:56:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3929 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:56:50 dist-test-slave-9cqd krb5kdc[23560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:56:50 dist-test-slave-9cqd krb5kdc[23560](info): set up 2 sockets
Sep 28 15:56:50 dist-test-slave-9cqd krb5kdc[23560](info): commencing operation
krb5kdc: starting...
W20260928 15:56:51.125738 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.005s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:56:51 dist-test-slave-9cqd krb5kdc[23560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:56:51.295861 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:40541
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:40541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:51.613057 23573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:51.613344 23573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:51.613514 23573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:51.620651 23573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:56:51.620780 23573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:51.620886 23573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:51.620981 23573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:56:51.621078 23573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:56:51.630812 23573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:40541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:40541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:51.632112 23573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:51.633432 23573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:51.645174 23579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:51.645244 23578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:51.645264 23581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:51.646265 23573 server_base.cc:1034] running on GCE node
I20260928 15:56:51.647012 23573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:51.648372 23573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:51.649577 23573 hybrid_clock.cc:648] HybridClock initialized: now 1790611011649520 us; error 46 us; skew 500 ppm
I20260928 15:56:51.649925 23573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:51 dist-test-slave-9cqd krb5kdc[23560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611011, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:51.652951 23573 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:56:51.654255 23573 webserver.cc:460] Webserver started at http://127.22.125.254:45935/ using document root <none> and password file <none>
I20260928 15:56:51.654870 23573 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:51.654965 23573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:51.655207 23573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:51.658016 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "dbcad04496364f2d85e4c5f706336c3f"
format_stamp: "Formatted at 2026-09-28 15:56:51 on dist-test-slave-9cqd"
server_key: "8ebda504b9378316c43e70dc9a3920b6"
server_key_iv: "ff0eafbc9fabadb5adf2c5e365c5aa8d"
server_key_version: "encryptionkey@0"
I20260928 15:56:51.658689 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "dbcad04496364f2d85e4c5f706336c3f"
format_stamp: "Formatted at 2026-09-28 15:56:51 on dist-test-slave-9cqd"
server_key: "8ebda504b9378316c43e70dc9a3920b6"
server_key_iv: "ff0eafbc9fabadb5adf2c5e365c5aa8d"
server_key_version: "encryptionkey@0"
I20260928 15:56:51.663543 23573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 15:56:51.666796 23588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:51.668002 23573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 15:56:51.668195 23573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "dbcad04496364f2d85e4c5f706336c3f"
format_stamp: "Formatted at 2026-09-28 15:56:51 on dist-test-slave-9cqd"
server_key: "8ebda504b9378316c43e70dc9a3920b6"
server_key_iv: "ff0eafbc9fabadb5adf2c5e365c5aa8d"
server_key_version: "encryptionkey@0"
I20260928 15:56:51.668350 23573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:51.703050 23573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:51.707412 23573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:51.707635 23573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:51.725461 23573 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:40541
I20260928 15:56:51.725490 23639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:40541 every 8 connection(s)
I20260928 15:56:51.726765 23573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:56:51.730361 23640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:51.731791 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23573
I20260928 15:56:51.732036 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:56:51.732709 23031 external_mini_cluster.cc:1428] Setting key a4978f2e931da93cee145af6b0130a9c
I20260928 15:56:51.742123 23640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f: Bootstrap starting.
I20260928 15:56:51.745111 23640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:51.746106 23640 log.cc:824] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f: Log is configured to *not* fsync() on all Append() calls
Sep 28 15:56:51 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:51.749125 23640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f: No bootstrap required, opened a new log
I20260928 15:56:51.754772 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:51.734543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34570 (local address 127.22.125.254:40541)
0928 15:56:51.735332 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:51.735346 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:51.735390 (+ 44us) server_negotiation.cc:407] Connection header received
0928 15:56:51.737084 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:51.737105 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:51.737469 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:51.737756 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:51.739579 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:51.741258 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:51.742749 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:51.743425 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:51.743615 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:51.747750 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:51.747757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:51.747770 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:51.747801 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:51.750828 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:51.751604 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:51.751609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:51.751615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:51.751778 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:51.752175 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:51.752178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:51.752180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:51.752666 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:51.752879 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:51.753428 (+ 549us) server_negotiation.cc:299] Negotiation successful
0928 15:56:51.753787 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":378,"threads_started":1}
I20260928 15:56:51.758061 23640 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcad04496364f2d85e4c5f706336c3f" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40541 } }
I20260928 15:56:51.758389 23640 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:51.758517 23640 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcad04496364f2d85e4c5f706336c3f, State: Initialized, Role: FOLLOWER
I20260928 15:56:51.759032 23640 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcad04496364f2d85e4c5f706336c3f" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40541 } }
I20260928 15:56:51.759262 23640 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:56:51.759428 23640 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:56:51.759585 23640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:51.761745 23640 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcad04496364f2d85e4c5f706336c3f" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40541 } }
I20260928 15:56:51.762220 23640 leader_election.cc:302] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbcad04496364f2d85e4c5f706336c3f; no voters:
I20260928 15:56:51.762748 23640 leader_election.cc:288] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:56:51.762848 23645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:56:51.763535 23645 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [term 1 LEADER]: Becoming Leader. State: Replica: dbcad04496364f2d85e4c5f706336c3f, State: Running, Role: LEADER
I20260928 15:56:51.764163 23645 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcad04496364f2d85e4c5f706336c3f" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40541 } }
I20260928 15:56:51.764854 23640 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:56:51.766721 23646 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbcad04496364f2d85e4c5f706336c3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcad04496364f2d85e4c5f706336c3f" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40541 } } }
I20260928 15:56:51.767138 23646 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [sys.catalog]: This master's current role is: LEADER
I20260928 15:56:51.766691 23647 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbcad04496364f2d85e4c5f706336c3f. Latest consensus state: current_term: 1 leader_uuid: "dbcad04496364f2d85e4c5f706336c3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbcad04496364f2d85e4c5f706336c3f" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40541 } } }
I20260928 15:56:51.767647 23647 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f [sys.catalog]: This master's current role is: LEADER
W20260928 15:56:51.776263 23661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 15:56:51.776360 23661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 15:56:51.776502 23651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:56:51.777971 23651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:56:51.785167 23651 catalog_manager.cc:1347] Generated new cluster ID: 7aba02c5d4cb425ca1d5285244dea9ae
I20260928 15:56:51.785269 23651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:56:51.798265 23651 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:56:51.799480 23651 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:56:51.807256 23651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbcad04496364f2d85e4c5f706336c3f: Generated new TSK 0
I20260928 15:56:51.808033 23651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:56:51.849725 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40541
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:52.181710 23666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:52.182020 23666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:52.182204 23666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:52.189949 23666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:52.190101 23666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:52.190274 23666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:56:52.199945 23666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40541
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:52.201249 23666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:52.202587 23666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:52.215526 23672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:52.215620 23671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:52.215639 23674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:52.216441 23666 server_base.cc:1034] running on GCE node
I20260928 15:56:52.217010 23666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:52.217888 23666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:52.219105 23666 hybrid_clock.cc:648] HybridClock initialized: now 1790611012219048 us; error 52 us; skew 500 ppm
I20260928 15:56:52.219408 23666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:52 dist-test-slave-9cqd krb5kdc[23560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:52.222414 23666 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:56:52.223820 23666 webserver.cc:460] Webserver started at http://127.22.125.193:34895/ using document root <none> and password file <none>
I20260928 15:56:52.224517 23666 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:52.224658 23666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:52.225000 23666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:52.227741 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae"
format_stamp: "Formatted at 2026-09-28 15:56:52 on dist-test-slave-9cqd"
server_key: "0883d7d9c196ca63815e7b291b51d6f7"
server_key_iv: "04157c28e65c5ae9fe455354c2d7f453"
server_key_version: "encryptionkey@0"
I20260928 15:56:52.228525 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae"
format_stamp: "Formatted at 2026-09-28 15:56:52 on dist-test-slave-9cqd"
server_key: "0883d7d9c196ca63815e7b291b51d6f7"
server_key_iv: "04157c28e65c5ae9fe455354c2d7f453"
server_key_version: "encryptionkey@0"
I20260928 15:56:52.233460 23666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260928 15:56:52.236769 23681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:52.237986 23666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:56:52.238153 23666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae"
format_stamp: "Formatted at 2026-09-28 15:56:52 on dist-test-slave-9cqd"
server_key: "0883d7d9c196ca63815e7b291b51d6f7"
server_key_iv: "04157c28e65c5ae9fe455354c2d7f453"
server_key_version: "encryptionkey@0"
I20260928 15:56:52.238364 23666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:52.252168 23666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:52.256421 23666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:52.256695 23666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:52.257593 23666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:56:52.258821 23666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:52.258919 23666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:52.259055 23666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:52.259132 23666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:52.291870 23666 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:43289
I20260928 15:56:52.291908 23793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:43289 every 8 connection(s)
I20260928 15:56:52.293129 23666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:56:52.296301 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23666
I20260928 15:56:52.296533 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:56:52.297328 23031 external_mini_cluster.cc:1428] Setting key 22a9fdf3ebbce049ab745103317bfcdd
Sep 28 15:56:52 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:56:52.314199 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:52.295596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:45357 (local address 127.22.125.254:40541)
0928 15:56:52.296099 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:52.296105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:52.297158 (+ 1053us) server_negotiation.cc:407] Connection header received
0928 15:56:52.299144 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:52.299166 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:52.299247 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:52.299369 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:52.300859 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.302036 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.303766 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.304220 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.304323 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:52.308559 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:52.308565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:52.308568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:52.308598 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:52.311189 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:52.311863 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:52.311873 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:52.311875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:52.311987 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:52.312428 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:52.312433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:52.312436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:52.312785 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:52.312936 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:52.313599 (+ 663us) server_negotiation.cc:299] Negotiation successful
0928 15:56:52.313828 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":249,"threads_started":1}
I20260928 15:56:52.315189 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:52.296176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:40541 (local address 127.22.125.193:45357)
0928 15:56:52.297005 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:52.297042 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:52.298846 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:52.299570 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:52.299578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:52.300125 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:52.300687 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:52.300704 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.302188 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.302191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:52.303614 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:52.303631 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.304459 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.304466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:52.304809 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:52.305512 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:52.305545 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:52.308346 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:52.311327 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:52.311335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:52.311350 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:52.311738 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:52.312114 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:52.312120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:52.312123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:52.312309 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:52.312909 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:52.312917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:52.313237 (+ 320us) client_negotiation.cc:769] Sending connection context
0928 15:56:52.313507 (+ 270us) client_negotiation.cc:240] Negotiation successful
0928 15:56:52.314001 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":383,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:56:52.316840 23794 heartbeater.cc:347] Connected to a master server at 127.22.125.254:40541
I20260928 15:56:52.317143 23794 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:52.317847 23794 heartbeater.cc:511] Master 127.22.125.254:40541 requested a full tablet report, sending...
I20260928 15:56:52.319832 23605 ts_manager.cc:194] Registered new tserver with Master: 4308eb98449b4d7fbcee3ee4acdaa4ae (127.22.125.193:43289)
I20260928 15:56:52.321596 23605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:45357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:56:52.339234 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40541
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:52.656617 23800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:52.656881 23800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:52.657068 23800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:52.664130 23800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:52.664278 23800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:52.664502 23800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:56:52.674497 23800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40541
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:52.675815 23800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:52.677132 23800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:52.689803 23808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:52.689932 23805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:52.690467 23806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:52.691058 23800 server_base.cc:1034] running on GCE node
I20260928 15:56:52.691663 23800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:52.692667 23800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:52.693889 23800 hybrid_clock.cc:648] HybridClock initialized: now 1790611012693830 us; error 49 us; skew 500 ppm
I20260928 15:56:52.694221 23800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:52 dist-test-slave-9cqd krb5kdc[23560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:52.702167 23800 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:56:52.703596 23800 webserver.cc:460] Webserver started at http://127.22.125.194:41347/ using document root <none> and password file <none>
I20260928 15:56:52.704347 23800 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:52.704496 23800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:52.704819 23800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:52.707664 23800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e"
format_stamp: "Formatted at 2026-09-28 15:56:52 on dist-test-slave-9cqd"
server_key: "5afaddb5a47c25257c358eb8edb4cc4b"
server_key_iv: "eb5473bdf1afd01d42097c0bbffdef67"
server_key_version: "encryptionkey@0"
I20260928 15:56:52.708541 23800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e"
format_stamp: "Formatted at 2026-09-28 15:56:52 on dist-test-slave-9cqd"
server_key: "5afaddb5a47c25257c358eb8edb4cc4b"
server_key_iv: "eb5473bdf1afd01d42097c0bbffdef67"
server_key_version: "encryptionkey@0"
I20260928 15:56:52.713518 23800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 15:56:52.716768 23815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:52.717965 23800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:56:52.718140 23800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e"
format_stamp: "Formatted at 2026-09-28 15:56:52 on dist-test-slave-9cqd"
server_key: "5afaddb5a47c25257c358eb8edb4cc4b"
server_key_iv: "eb5473bdf1afd01d42097c0bbffdef67"
server_key_version: "encryptionkey@0"
I20260928 15:56:52.718334 23800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:52.741812 23800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:52.745873 23800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:52.746186 23800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:52.747118 23800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:56:52.748451 23800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:52.748553 23800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:52.748704 23800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:52.748783 23800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:52.782542 23800 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:36775
I20260928 15:56:52.782635 23927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:36775 every 8 connection(s)
I20260928 15:56:52.783866 23800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:56:52.784463 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23800
I20260928 15:56:52.784682 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:56:52.785300 23031 external_mini_cluster.cc:1428] Setting key 70d0f79f8e560f0f561fa492c79ee661
Sep 28 15:56:52 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611012, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
I20260928 15:56:52.805389 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:52.787088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:44127 (local address 127.22.125.254:40541)
0928 15:56:52.787256 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:52.787261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:52.788732 (+ 1471us) server_negotiation.cc:407] Connection header received
0928 15:56:52.790621 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:52.790627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:52.790725 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:52.790836 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:52.792371 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.793078 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.794603 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.795012 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.795144 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:52.799310 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:52.799315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:52.799318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:52.799361 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:52.802048 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:52.802781 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:52.802785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:52.802787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:52.802922 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:52.803373 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:52.803381 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:52.803389 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:52.803678 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:52.803809 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:52.804688 (+ 879us) server_negotiation.cc:299] Negotiation successful
0928 15:56:52.804947 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 15:56:52.806069 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:52.787723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:40541 (local address 127.22.125.194:44127)
0928 15:56:52.788575 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:52.788620 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:52.790354 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:52.791072 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:52.791080 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:52.791578 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:52.792208 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:52.792228 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.793263 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.793267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:52.794452 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:52.794464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:52.795243 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:52.795252 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:52.795483 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:52.796338 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:52.796360 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:52.799068 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:52.802216 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:52.802223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:52.802236 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:52.802659 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:52.803019 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:52.803022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:52.803025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:52.803205 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:52.803786 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:52.803791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:52.804192 (+ 401us) client_negotiation.cc:769] Sending connection context
0928 15:56:52.804542 (+ 350us) client_negotiation.cc:240] Negotiation successful
0928 15:56:52.804940 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":377,"threads_started":1}
I20260928 15:56:52.807636 23928 heartbeater.cc:347] Connected to a master server at 127.22.125.254:40541
I20260928 15:56:52.807956 23928 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:52.808694 23928 heartbeater.cc:511] Master 127.22.125.254:40541 requested a full tablet report, sending...
I20260928 15:56:52.810137 23605 ts_manager.cc:194] Registered new tserver with Master: 8cfffaf90fb54f89a55145fa0d9b7a4e (127.22.125.194:36775)
I20260928 15:56:52.811044 23605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:44127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:56:52.825492 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40541
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:53.138479 23933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:53.138804 23933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:53.138974 23933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:53.146341 23933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:53.146489 23933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:53.146669 23933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:56:53.156178 23933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40541
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:53.157416 23933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:53.158713 23933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:53.171564 23939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:53.171680 23938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:53.172088 23941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:53.172624 23933 server_base.cc:1034] running on GCE node
I20260928 15:56:53.173240 23933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:53.174031 23933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:53.175261 23933 hybrid_clock.cc:648] HybridClock initialized: now 1790611013175211 us; error 51 us; skew 500 ppm
I20260928 15:56:53.175617 23933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:53.178791 23933 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:56:53.180241 23933 webserver.cc:460] Webserver started at http://127.22.125.195:46345/ using document root <none> and password file <none>
I20260928 15:56:53.180828 23933 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:53.180925 23933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:53.181171 23933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:53.183796 23933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "460343bbbf0048c6b38b00028dab0f1f"
format_stamp: "Formatted at 2026-09-28 15:56:53 on dist-test-slave-9cqd"
server_key: "a06c798993dc50f147d433af873ac607"
server_key_iv: "b1f7c72baa5bb3033faf0260c71b4073"
server_key_version: "encryptionkey@0"
I20260928 15:56:53.184573 23933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "460343bbbf0048c6b38b00028dab0f1f"
format_stamp: "Formatted at 2026-09-28 15:56:53 on dist-test-slave-9cqd"
server_key: "a06c798993dc50f147d433af873ac607"
server_key_iv: "b1f7c72baa5bb3033faf0260c71b4073"
server_key_version: "encryptionkey@0"
I20260928 15:56:53.189389 23933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260928 15:56:53.192700 23948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:53.193928 23933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:56:53.194049 23933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "460343bbbf0048c6b38b00028dab0f1f"
format_stamp: "Formatted at 2026-09-28 15:56:53 on dist-test-slave-9cqd"
server_key: "a06c798993dc50f147d433af873ac607"
server_key_iv: "b1f7c72baa5bb3033faf0260c71b4073"
server_key_version: "encryptionkey@0"
I20260928 15:56:53.194199 23933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:53.218298 23933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:53.222818 23933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:53.223106 23933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:53.223986 23933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:56:53.225292 23933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:53.225365 23933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:53.225531 23933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:53.225579 23933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:53.258697 23933 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:44367
I20260928 15:56:53.258754 24060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:44367 every 8 connection(s)
I20260928 15:56:53.260097 23933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:56:53.260406 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 23933
I20260928 15:56:53.260603 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:56:53.261137 23031 external_mini_cluster.cc:1428] Setting key 8a4653a3b9f67adb6dfe1985ad10ec2d
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:53.280229 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.262783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:33329 (local address 127.22.125.254:40541)
0928 15:56:53.262987 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.262993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.264564 (+ 1571us) server_negotiation.cc:407] Connection header received
0928 15:56:53.266466 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.266470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.266562 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.266648 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:53.268094 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.268878 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.270323 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.270797 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.270942 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.274337 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:53.274341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:53.274345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:53.274395 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:53.277648 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.278309 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.278313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.278315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.278429 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.278784 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.278787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.278788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.279050 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:53.279157 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.279757 (+ 600us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.279906 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 15:56:53.281203 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.263440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:40541 (local address 127.22.125.195:33329)
0928 15:56:53.264417 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:53.264460 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:53.266216 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:53.266838 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:53.266846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:53.267346 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:53.267909 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.267926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.269007 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.269011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.270195 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.270207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.271005 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.271010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.271204 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.271848 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:53.271869 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:53.274142 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:53.277764 (+ 3622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:53.277771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:53.277782 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:53.278193 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:53.278521 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:53.278524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:53.278526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:53.278690 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:53.279156 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:53.279161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:53.279454 (+ 293us) client_negotiation.cc:769] Sending connection context
0928 15:56:53.279681 (+ 227us) client_negotiation.cc:240] Negotiation successful
0928 15:56:53.280106 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":335,"threads_started":1}
I20260928 15:56:53.282505 24061 heartbeater.cc:347] Connected to a master server at 127.22.125.254:40541
I20260928 15:56:53.282773 24061 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:53.283424 24061 heartbeater.cc:511] Master 127.22.125.254:40541 requested a full tablet report, sending...
I20260928 15:56:53.284765 23605 ts_manager.cc:194] Registered new tserver with Master: 460343bbbf0048c6b38b00028dab0f1f (127.22.125.195:44367)
I20260928 15:56:53.285578 23605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:33329
I20260928 15:56:53.289049 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:56:53.311376 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.299549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.22.125.254:40541)
0928 15:56:53.299708 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.299713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.300282 (+ 569us) server_negotiation.cc:407] Connection header received
0928 15:56:53.300414 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.300418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.300476 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.300560 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:53.301954 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.302675 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.303890 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.304441 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.304536 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.306715 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:53.306724 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:53.306730 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:53.306772 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:53.309176 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.309774 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.309778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.309780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.309908 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.310211 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.310214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.310215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.310512 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:53.310679 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.310974 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.311119 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 15:56:53.324532 23794 heartbeater.cc:503] Master 127.22.125.254:40541 was elected leader, sending a full tablet report...
I20260928 15:56:53.324288 23605 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:56:53.341252 23605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:56:53.392035 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.371252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:36775 (local address 127.0.0.1:50646)
0928 15:56:53.384491 (+ 13239us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:53.384510 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:53.384627 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:53.385243 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:53.385247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:53.385271 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:53.385702 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.385714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.387901 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.387906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.389689 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.389700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.391167 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.391174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.391341 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.391378 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 15:56:53.391480 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 15:56:53.391611 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13068,"mutex_wait_us":1,"thread_start_us":12889,"threads_started":1}
I20260928 15:56:53.392288 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.371233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.22.125.193:43289)
0928 15:56:53.381871 (+ 10638us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.381883 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.382807 (+ 924us) server_negotiation.cc:407] Connection header received
0928 15:56:53.383248 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.383254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.383473 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.383683 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:53.384528 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.386207 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.389018 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.390572 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.390764 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.390977 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.391744 (+ 767us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.391921 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10439,"thread_start_us":10284,"threads_started":1}
I20260928 15:56:53.392464 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.371380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.22.125.194:36775)
0928 15:56:53.382211 (+ 10831us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.382221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.384620 (+ 2399us) server_negotiation.cc:407] Connection header received
0928 15:56:53.384763 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.384773 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.385030 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.385898 (+ 868us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:53.386136 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.387752 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.389853 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.390887 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.391111 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.391276 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.391583 (+ 307us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.391767 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10652,"thread_start_us":10848,"threads_started":1}
I20260928 15:56:53.392798 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.370967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:43289 (local address 127.0.0.1:60826)
0928 15:56:53.382605 (+ 11638us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:53.382657 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:53.382977 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:53.383692 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:53.383700 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:53.383728 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:53.384342 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.384354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.386379 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.386386 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.388840 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.388862 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.390788 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.390797 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.391095 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.391163 (+ 68us) client_negotiation.cc:769] Sending connection context
0928 15:56:53.391284 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 15:56:53.391483 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11295,"mutex_wait_us":56,"thread_start_us":10992,"threads_started":1}
I20260928 15:56:53.393944 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.385478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44367 (local address 127.0.0.1:40222)
0928 15:56:53.386043 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:53.386056 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:53.386203 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:53.386682 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:53.386687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:53.386710 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:53.387145 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.387158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.388927 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.388931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.391013 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.391026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.393229 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.393233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.393342 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.393361 (+ 19us) client_negotiation.cc:769] Sending connection context
0928 15:56:53.393463 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 15:56:53.393584 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":1,"thread_start_us":341,"threads_started":1}
I20260928 15:56:53.394024 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.384819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.22.125.195:44367)
0928 15:56:53.385490 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.385498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.386156 (+ 658us) server_negotiation.cc:407] Connection header received
0928 15:56:53.386357 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.386363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.386518 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.386689 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:53.387389 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.388768 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.391195 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.392703 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.392862 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.393051 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.393527 (+ 476us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.393690 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":361,"threads_started":1}
I20260928 15:56:53.396854 23863 tablet_service.cc:1467] Processing CreateTablet for tablet b3ad6c4114d148389a6b182fe77248b2 (DEFAULT_TABLE table=test-table [id=41ebabcc42de4a2997e30014a04ee6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:53.397153 23729 tablet_service.cc:1467] Processing CreateTablet for tablet b3ad6c4114d148389a6b182fe77248b2 (DEFAULT_TABLE table=test-table [id=41ebabcc42de4a2997e30014a04ee6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:53.397445 23996 tablet_service.cc:1467] Processing CreateTablet for tablet b3ad6c4114d148389a6b182fe77248b2 (DEFAULT_TABLE table=test-table [id=41ebabcc42de4a2997e30014a04ee6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:53.398454 23863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ad6c4114d148389a6b182fe77248b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:53.398545 23729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ad6c4114d148389a6b182fe77248b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:53.398839 23996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ad6c4114d148389a6b182fe77248b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:53.409706 24079 tablet_bootstrap.cc:492] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f: Bootstrap starting.
I20260928 15:56:53.413102 24080 tablet_bootstrap.cc:492] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae: Bootstrap starting.
I20260928 15:56:53.413538 24079 tablet_bootstrap.cc:654] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:53.415798 24079 log.cc:824] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:53.416643 24080 tablet_bootstrap.cc:654] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:53.417310 24081 tablet_bootstrap.cc:492] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e: Bootstrap starting.
I20260928 15:56:53.417961 24080 log.cc:824] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:53.420473 24080 tablet_bootstrap.cc:492] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae: No bootstrap required, opened a new log
I20260928 15:56:53.420548 24079 tablet_bootstrap.cc:492] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f: No bootstrap required, opened a new log
I20260928 15:56:53.420715 24081 tablet_bootstrap.cc:654] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:53.420792 24080 ts_tablet_manager.cc:1397] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 15:56:53.420864 24079 ts_tablet_manager.cc:1397] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260928 15:56:53.421922 24081 log.cc:824] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:53.425022 24081 tablet_bootstrap.cc:492] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e: No bootstrap required, opened a new log
I20260928 15:56:53.425302 24081 ts_tablet_manager.cc:1397] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260928 15:56:53.428982 24080 raft_consensus.cc:348] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.429342 24080 raft_consensus.cc:374] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:53.429455 24080 raft_consensus.cc:729] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4308eb98449b4d7fbcee3ee4acdaa4ae, State: Initialized, Role: FOLLOWER
I20260928 15:56:53.430056 24080 consensus_queue.cc:260] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.431587 24080 ts_tablet_manager.cc:1428] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260928 15:56:53.432108 24079 raft_consensus.cc:348] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.432487 24079 raft_consensus.cc:374] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:53.432621 24079 raft_consensus.cc:729] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460343bbbf0048c6b38b00028dab0f1f, State: Initialized, Role: FOLLOWER
I20260928 15:56:53.433189 24079 consensus_queue.cc:260] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.435730 24081 raft_consensus.cc:348] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.436187 24081 raft_consensus.cc:374] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:53.436288 24081 raft_consensus.cc:729] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfffaf90fb54f89a55145fa0d9b7a4e, State: Initialized, Role: FOLLOWER
I20260928 15:56:53.436595 24079 ts_tablet_manager.cc:1428] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f: Time spent starting tablet: real 0.016s user 0.006s sys 0.008s
I20260928 15:56:53.436851 24081 consensus_queue.cc:260] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.437076 24061 heartbeater.cc:503] Master 127.22.125.254:40541 was elected leader, sending a full tablet report...
I20260928 15:56:53.437951 23928 heartbeater.cc:503] Master 127.22.125.254:40541 was elected leader, sending a full tablet report...
I20260928 15:56:53.438791 24081 ts_tablet_manager.cc:1428] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260928 15:56:53.512157 24062 tablet.cc:2369] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:56:53.536182 23929 tablet.cc:2369] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:56:53.545405 23795 tablet.cc:2369] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:56:53.598913 24085 raft_consensus.cc:482] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:56:53.599350 24085 raft_consensus.cc:504] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.601778 24085 leader_election.cc:288] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cfffaf90fb54f89a55145fa0d9b7a4e (127.22.125.194:36775), 460343bbbf0048c6b38b00028dab0f1f (127.22.125.195:44367)
I20260928 15:56:53.608510 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.602201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:50163 (local address 127.22.125.194:36775)
0928 15:56:53.602405 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.602410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.602836 (+ 426us) server_negotiation.cc:407] Connection header received
0928 15:56:53.603048 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.603055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.603150 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.603259 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:53.604177 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.604925 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.606730 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.607534 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.607722 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.607798 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.608104 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.608235 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 15:56:53.608676 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.602025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:36775 (local address 127.22.125.193:50163)
0928 15:56:53.602720 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:53.602742 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:53.602884 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:53.603381 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:53.603387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:53.603430 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:53.603738 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.603746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.605059 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.605067 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.606561 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.606573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.607748 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.607755 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.607908 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.607935 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 15:56:53.608051 (+ 116us) client_negotiation.cc:240] Negotiation successful
0928 15:56:53.608222 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":1,"thread_start_us":378,"threads_started":1}
I20260928 15:56:53.608812 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.603021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:58191 (local address 127.22.125.195:44367)
0928 15:56:53.603284 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.603290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.603686 (+ 396us) server_negotiation.cc:407] Connection header received
0928 15:56:53.603906 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.603910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.603969 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.604092 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:53.604636 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.605354 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.607077 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.607823 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.608013 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.608120 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.608422 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.608550 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 15:56:53.608961 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.602851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44367 (local address 127.22.125.193:58191)
0928 15:56:53.603574 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:53.603591 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:53.603754 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:53.604102 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:53.604106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:53.604128 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:53.604481 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.604490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.605502 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.605507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.606912 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:53.606925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.608105 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.608112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:53.608264 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.608287 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 15:56:53.608405 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 15:56:53.608530 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":371,"threads_started":1}
I20260928 15:56:53.609826 24016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ad6c4114d148389a6b182fe77248b2" candidate_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460343bbbf0048c6b38b00028dab0f1f" is_pre_election: true
I20260928 15:56:53.609823 23883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ad6c4114d148389a6b182fe77248b2" candidate_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" is_pre_election: true
I20260928 15:56:53.610320 24016 raft_consensus.cc:2454] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4308eb98449b4d7fbcee3ee4acdaa4ae in term 0.
I20260928 15:56:53.610360 23883 raft_consensus.cc:2454] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4308eb98449b4d7fbcee3ee4acdaa4ae in term 0.
I20260928 15:56:53.611114 23685 leader_election.cc:302] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4308eb98449b4d7fbcee3ee4acdaa4ae, 460343bbbf0048c6b38b00028dab0f1f; no voters:
I20260928 15:56:53.611522 24085 raft_consensus.cc:2781] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:56:53.611677 24085 raft_consensus.cc:482] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:56:53.611802 24085 raft_consensus.cc:3037] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:53.614070 24085 raft_consensus.cc:504] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.614672 24085 leader_election.cc:288] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [CANDIDATE]: Term 1 election: Requested vote from peers 8cfffaf90fb54f89a55145fa0d9b7a4e (127.22.125.194:36775), 460343bbbf0048c6b38b00028dab0f1f (127.22.125.195:44367)
I20260928 15:56:53.615207 23883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ad6c4114d148389a6b182fe77248b2" candidate_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e"
I20260928 15:56:53.615270 24016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3ad6c4114d148389a6b182fe77248b2" candidate_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460343bbbf0048c6b38b00028dab0f1f"
I20260928 15:56:53.615413 23883 raft_consensus.cc:3037] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:53.615478 24016 raft_consensus.cc:3037] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:53.617668 23883 raft_consensus.cc:2454] T b3ad6c4114d148389a6b182fe77248b2 P 8cfffaf90fb54f89a55145fa0d9b7a4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4308eb98449b4d7fbcee3ee4acdaa4ae in term 1.
I20260928 15:56:53.617677 24016 raft_consensus.cc:2454] T b3ad6c4114d148389a6b182fe77248b2 P 460343bbbf0048c6b38b00028dab0f1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4308eb98449b4d7fbcee3ee4acdaa4ae in term 1.
I20260928 15:56:53.618036 23685 leader_election.cc:302] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4308eb98449b4d7fbcee3ee4acdaa4ae, 8cfffaf90fb54f89a55145fa0d9b7a4e; no voters:
I20260928 15:56:53.618325 24085 raft_consensus.cc:2781] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:56:53.618778 24085 raft_consensus.cc:686] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [term 1 LEADER]: Becoming Leader. State: Replica: 4308eb98449b4d7fbcee3ee4acdaa4ae, State: Running, Role: LEADER
I20260928 15:56:53.619371 24085 consensus_queue.cc:237] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } }
I20260928 15:56:53.628244 23605 catalog_manager.cc:5495] T b3ad6c4114d148389a6b182fe77248b2 P 4308eb98449b4d7fbcee3ee4acdaa4ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 4308eb98449b4d7fbcee3ee4acdaa4ae (127.22.125.193). New cstate: current_term: 1 leader_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4308eb98449b4d7fbcee3ee4acdaa4ae" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 43289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cfffaf90fb54f89a55145fa0d9b7a4e" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "460343bbbf0048c6b38b00028dab0f1f" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44367 } health_report { overall_health: UNKNOWN } } }
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:56:53.675233 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.660327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.22.125.193:43289)
0928 15:56:53.660536 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.660543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.660990 (+ 447us) server_negotiation.cc:407] Connection header received
0928 15:56:53.661205 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.661210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.661288 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.661394 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:53.662897 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.663606 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.664925 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.665380 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.665509 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.669489 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:53.669505 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:53.669521 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:53.669555 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:53.672706 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.673345 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.673354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.673356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.673524 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.673896 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.673899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.673901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.674234 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:53.674455 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.674694 (+ 239us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.674873 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:56:53.721165 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.707199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60846 (local address 127.22.125.193:43289)
0928 15:56:53.707471 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.707479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.707761 (+ 282us) server_negotiation.cc:407] Connection header received
0928 15:56:53.707983 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.707987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.708103 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.708210 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:53.709569 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.710295 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.711543 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.712162 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.712297 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.716233 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:53.716238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:53.716241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:53.716275 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:53.718853 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.719363 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.719367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.719369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.719503 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.719813 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.719816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.719818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.720113 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:53.720399 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.720642 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.720811 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:56:53.765558 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.751871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.22.125.193:43289)
0928 15:56:53.752134 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.752142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.752456 (+ 314us) server_negotiation.cc:407] Connection header received
0928 15:56:53.752673 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.752677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.752757 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.752864 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:53.754237 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.755005 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.756369 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.756800 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.756943 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.760726 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:53.760731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:53.760734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:53.760770 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:53.763137 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.763767 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.763771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.763774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.763908 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.764283 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.764286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.764288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.764562 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:53.764750 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.764991 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.765239 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260928 15:56:53.767570 23709 tablet_service.cc:2908] Rejecting scan request for tablet b3ad6c4114d148389a6b182fe77248b2: Uninitialized: clean time has not yet been initialized
I20260928 15:56:53.771847 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23666
I20260928 15:56:53.784552 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.772793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.22.125.193:43289)
0928 15:56:53.772928 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.772932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.773358 (+ 426us) server_negotiation.cc:407] Connection header received
0928 15:56:53.773498 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.773502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.773557 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.773632 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:53.774760 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.775438 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.776639 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.777032 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.777121 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.779268 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:53.779272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:53.779275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:53.779300 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:53.781870 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.782656 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.782659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.782661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.782781 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.783179 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.783182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.783184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.783448 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:53.783626 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.783986 (+ 360us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.784236 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 15:56:53.785202 23789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:53.882488 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23666
I20260928 15:56:53.899180 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23800
Sep 28 15:56:53 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:56:53.912986 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:53.900000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.22.125.194:36775)
0928 15:56:53.900216 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:53.900220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:53.900237 (+ 17us) server_negotiation.cc:407] Connection header received
0928 15:56:53.900320 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:53.900324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:53.900403 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:53.900485 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:53.901549 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.902299 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.903483 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:53.903856 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:53.903948 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:53.907900 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:53.907912 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:53.907927 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:53.907956 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:53.910549 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.911129 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.911135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.911137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.911252 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:53.911563 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:53.911566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:53.911568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:53.911898 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:53.912201 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:53.912484 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 15:56:53.912639 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260928 15:56:53.913620 23923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:54.001571 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23800
I20260928 15:56:54.017518 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23933
Sep 28 15:56:54 dist-test-slave-9cqd krb5kdc[23560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:56:54.031522 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:54.018329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.22.125.195:44367)
0928 15:56:54.018504 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:54.018509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:54.018527 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:56:54.018657 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:54.018661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:54.018735 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:54.018818 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:54.020031 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:54.020777 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:54.022016 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:54.022374 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:54.022463 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:54.026460 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:54.026473 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:54.026488 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:54.026518 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:54.029077 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:54.029710 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:54.029717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:54.029719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:54.029863 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:54.030190 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:54.030193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:54.030195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:54.030547 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:54.030774 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:54.030964 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 15:56:54.031129 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 15:56:54.032236 24056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:54.120586 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23933
I20260928 15:56:54.137092 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 23573
I20260928 15:56:54.138306 23635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:54.245244 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 23573
2026-09-28T15:56:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4311 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:56:54 dist-test-slave-9cqd krb5kdc[24109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:56:54 dist-test-slave-9cqd krb5kdc[24109](info): set up 2 sockets
Sep 28 15:56:54 dist-test-slave-9cqd krb5kdc[24109](info): commencing operation
krb5kdc: starting...
W20260928 15:56:55.325733 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.005s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:56:55 dist-test-slave-9cqd krb5kdc[24109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:56:55.506006 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:36909
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:36909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:55.818038 24122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:55.818315 24122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:55.818403 24122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:55.825418 24122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:56:55.825507 24122 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 15:56:55.825603 24122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:55.825704 24122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:55.825793 24122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:56:55.825896 24122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:56:55.835469 24122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:36909
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:36909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:55.836791 24122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:55.838070 24122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:55.849802 24128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:55.849774 24130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:55.849759 24127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:55.850755 24122 server_base.cc:1034] running on GCE node
I20260928 15:56:55.851526 24122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:55.852921 24122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:55.854202 24122 hybrid_clock.cc:648] HybridClock initialized: now 1790611015854138 us; error 36 us; skew 500 ppm
I20260928 15:56:55.854561 24122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:55 dist-test-slave-9cqd krb5kdc[24109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:55.857597 24122 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:56:55.859019 24122 webserver.cc:460] Webserver started at http://127.22.125.254:40017/ using document root <none> and password file <none>
I20260928 15:56:55.859694 24122 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:55.859827 24122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:55.860206 24122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:55.862975 24122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "476dcdf34e194656b1dec9e90da8b5d6"
format_stamp: "Formatted at 2026-09-28 15:56:55 on dist-test-slave-9cqd"
server_key: "9b910cffd8aa9ba0dc0b1fd62f2abb78"
server_key_iv: "68ea669947377e874be514a7a2262f44"
server_key_version: "encryptionkey@0"
I20260928 15:56:55.863740 24122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "476dcdf34e194656b1dec9e90da8b5d6"
format_stamp: "Formatted at 2026-09-28 15:56:55 on dist-test-slave-9cqd"
server_key: "9b910cffd8aa9ba0dc0b1fd62f2abb78"
server_key_iv: "68ea669947377e874be514a7a2262f44"
server_key_version: "encryptionkey@0"
I20260928 15:56:55.868816 24122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260928 15:56:55.871976 24137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:55.873255 24122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 15:56:55.873448 24122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "476dcdf34e194656b1dec9e90da8b5d6"
format_stamp: "Formatted at 2026-09-28 15:56:55 on dist-test-slave-9cqd"
server_key: "9b910cffd8aa9ba0dc0b1fd62f2abb78"
server_key_iv: "68ea669947377e874be514a7a2262f44"
server_key_version: "encryptionkey@0"
I20260928 15:56:55.873664 24122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:55.895886 24122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:55.899576 24122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:55.899835 24122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:55.921485 24122 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:36909
I20260928 15:56:55.921495 24198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:36909 every 8 connection(s)
I20260928 15:56:55.922793 24122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:56:55.926430 24199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:55.931643 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 24122
I20260928 15:56:55.931872 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:56:55.932741 23031 external_mini_cluster.cc:1428] Setting key b1bb26d5f280b18af62135fc05009152
I20260928 15:56:55.937594 24199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6: Bootstrap starting.
I20260928 15:56:55.940572 24199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:55.941555 24199 log.cc:824] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:55.944684 24199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6: No bootstrap required, opened a new log
Sep 28 15:56:55 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:55.954736 24199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476dcdf34e194656b1dec9e90da8b5d6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36909 } }
I20260928 15:56:55.955158 24199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:55.955276 24199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 476dcdf34e194656b1dec9e90da8b5d6, State: Initialized, Role: FOLLOWER
I20260928 15:56:55.955912 24199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476dcdf34e194656b1dec9e90da8b5d6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36909 } }
I20260928 15:56:55.956190 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:55.934851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.22.125.254:36909)
0928 15:56:55.935646 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:55.935657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:55.935700 (+ 43us) server_negotiation.cc:407] Connection header received
0928 15:56:55.937369 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:55.937392 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:55.937824 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:55.938204 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:55.939610 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:55.941028 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:55.942478 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:55.943471 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:55.943657 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:55.947741 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:55.947752 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:55.947771 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:55.947820 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:55.951800 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:55.952504 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:55.952512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:55.952520 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:55.952745 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:55.953107 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:55.953114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:55.953118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:55.953786 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:55.954093 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:55.954348 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 15:56:55.954782 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":585,"thread_start_us":362,"threads_started":1}
I20260928 15:56:55.956226 24199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:56:55.956353 24199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:56:55.956539 24199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:55.959234 24199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476dcdf34e194656b1dec9e90da8b5d6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36909 } }
I20260928 15:56:55.959648 24199 leader_election.cc:302] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 476dcdf34e194656b1dec9e90da8b5d6; no voters:
I20260928 15:56:55.960206 24199 leader_election.cc:288] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:56:55.960319 24204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:56:55.961030 24204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [term 1 LEADER]: Becoming Leader. State: Replica: 476dcdf34e194656b1dec9e90da8b5d6, State: Running, Role: LEADER
I20260928 15:56:55.961591 24204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476dcdf34e194656b1dec9e90da8b5d6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36909 } }
I20260928 15:56:55.962237 24199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:56:55.964818 24206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 476dcdf34e194656b1dec9e90da8b5d6. Latest consensus state: current_term: 1 leader_uuid: "476dcdf34e194656b1dec9e90da8b5d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476dcdf34e194656b1dec9e90da8b5d6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36909 } } }
I20260928 15:56:55.965272 24206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [sys.catalog]: This master's current role is: LEADER
I20260928 15:56:55.964859 24205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "476dcdf34e194656b1dec9e90da8b5d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476dcdf34e194656b1dec9e90da8b5d6" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36909 } } }
I20260928 15:56:55.965756 24205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6 [sys.catalog]: This master's current role is: LEADER
I20260928 15:56:55.966781 24211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:56:55.972955 24211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:56:55.980746 24211 catalog_manager.cc:1347] Generated new cluster ID: 3a4b44d662a0496e931033957fa5937c
I20260928 15:56:55.980865 24211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:56:55.997084 24211 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:56:55.998368 24211 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:56:56.007920 24211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6: Generated new TSK 0
I20260928 15:56:56.008780 24211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:56:56.057219 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36909
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:56.370118 24225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:56.370431 24225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:56.370599 24225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:56.378044 24225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:56.378178 24225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:56.378329 24225 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:56:56.378468 24225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:56:56.387996 24225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:36909
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:56.389251 24225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:56.390589 24225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:56.403203 24230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:56.403228 24231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:56.403491 24233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:56.404428 24225 server_base.cc:1034] running on GCE node
I20260928 15:56:56.405015 24225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:56.405838 24225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:56.407047 24225 hybrid_clock.cc:648] HybridClock initialized: now 1790611016406966 us; error 60 us; skew 500 ppm
I20260928 15:56:56.407331 24225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:56 dist-test-slave-9cqd krb5kdc[24109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:56.410341 24225 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:56:56.411685 24225 webserver.cc:460] Webserver started at http://127.22.125.193:45749/ using document root <none> and password file <none>
I20260928 15:56:56.412424 24225 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:56.412550 24225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:56.412858 24225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:56.415645 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "a2e26707648940a2a35fd6c0bf5eba66"
format_stamp: "Formatted at 2026-09-28 15:56:56 on dist-test-slave-9cqd"
server_key: "fe2002c3386565d8370e0924fff36ccf"
server_key_iv: "780c519c2387d0333dc0e5f3da44bbee"
server_key_version: "encryptionkey@0"
I20260928 15:56:56.416458 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "a2e26707648940a2a35fd6c0bf5eba66"
format_stamp: "Formatted at 2026-09-28 15:56:56 on dist-test-slave-9cqd"
server_key: "fe2002c3386565d8370e0924fff36ccf"
server_key_iv: "780c519c2387d0333dc0e5f3da44bbee"
server_key_version: "encryptionkey@0"
I20260928 15:56:56.421271 24225 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260928 15:56:56.424513 24240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:56.425724 24225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260928 15:56:56.425894 24225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "a2e26707648940a2a35fd6c0bf5eba66"
format_stamp: "Formatted at 2026-09-28 15:56:56 on dist-test-slave-9cqd"
server_key: "fe2002c3386565d8370e0924fff36ccf"
server_key_iv: "780c519c2387d0333dc0e5f3da44bbee"
server_key_version: "encryptionkey@0"
I20260928 15:56:56.426101 24225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:56.439507 24225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:56.443221 24225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:56.443498 24225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:56.446303 24225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:56.446429 24225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:56.446587 24225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:56.446729 24225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 28 15:56:56 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:56.472630 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:56.447076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36909 (local address 127.22.125.193:41461)
0928 15:56:56.448060 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:56.448106 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:56.449943 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:56.450698 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:56.450706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:56.451477 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:56.452543 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:56.452575 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.453977 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.453986 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:56.456691 (+ 2705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:56.456704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.457716 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.457725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:56.458000 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:56.458893 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:56.458923 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:56.462064 (+ 3141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:56.467156 (+ 5092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:56.467169 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:56.467192 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:56.467923 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:56.468629 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:56.468634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:56.468638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:56.468914 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:56.469583 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:56.469591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:56.469969 (+ 378us) client_negotiation.cc:769] Sending connection context
0928 15:56:56.470288 (+ 319us) client_negotiation.cc:240] Negotiation successful
0928 15:56:56.470943 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":410,"threads_started":1}
I20260928 15:56:56.473567 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:56.446571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:41461 (local address 127.22.125.254:36909)
0928 15:56:56.446791 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:56.446799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:56.448225 (+ 1426us) server_negotiation.cc:407] Connection header received
0928 15:56:56.450236 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:56.450242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:56.450339 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:56.450456 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:56.452732 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.453805 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.456844 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.457447 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.457587 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:56.462330 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:56.462340 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:56.462348 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:56.462401 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:56.466941 (+ 4540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:56.468120 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:56.468127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:56.468130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:56.468398 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:56.469107 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:56.469114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:56.469120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:56.469446 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:56.469612 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:56.472785 (+ 3173us) server_negotiation.cc:299] Negotiation successful
0928 15:56:56.473097 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 15:56:56.497306 24225 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:37973
I20260928 15:56:56.497344 24359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:37973 every 8 connection(s)
I20260928 15:56:56.498560 24225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:56:56.501149 24360 heartbeater.cc:347] Connected to a master server at 127.22.125.254:36909
I20260928 15:56:56.501386 24360 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:56.502032 24360 heartbeater.cc:511] Master 127.22.125.254:36909 requested a full tablet report, sending...
I20260928 15:56:56.503988 24154 ts_manager.cc:194] Registered new tserver with Master: a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193:37973)
I20260928 15:56:56.505687 24154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:41461
I20260928 15:56:56.506160 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 24225
I20260928 15:56:56.506400 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:56:56.506714 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:56.492821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:49337 (local address 127.22.125.254:36909)
0928 15:56:56.492996 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:56.493000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:56.493737 (+ 737us) server_negotiation.cc:407] Connection header received
0928 15:56:56.493959 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:56.493965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:56.494078 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:56.494191 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:56.495664 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.496646 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.498131 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.498555 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.498704 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:56.500959 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:56.500969 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:56.500973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:56.501036 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:56.503865 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:56.504618 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:56.504624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:56.504628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:56.504751 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:56.505145 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:56.505150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:56.505153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:56.505516 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:56.505678 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:56.506096 (+ 418us) server_negotiation.cc:299] Negotiation successful
0928 15:56:56.506343 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 15:56:56.506964 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:56.493022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36909 (local address 127.22.125.193:49337)
0928 15:56:56.493622 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:56.493637 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:56.493818 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:56.494253 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:56.494261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:56.495069 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:56.495470 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:56.495484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.496803 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.496813 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:56.497985 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:56.497998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.498749 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.498756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:56.498949 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:56.499709 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:56.499738 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:56.500756 (+ 1018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:56.504013 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:56.504021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:56.504025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:56.504491 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:56.504866 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:56.504872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:56.504876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:56.505037 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:56.505640 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:56.505645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:56.505849 (+ 204us) client_negotiation.cc:769] Sending connection context
0928 15:56:56.506005 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 15:56:56.506330 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":342,"threads_started":1}
I20260928 15:56:56.507282 23031 external_mini_cluster.cc:1428] Setting key d40a28e9124f4ff21d24230ed5d946e5
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:56:56.547493 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36909
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:56.864116 24364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:56.864388 24364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:56.864482 24364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:56.871611 24364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:56.871708 24364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:56.871795 24364 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:56:56.871874 24364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:56:56.881861 24364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:36909
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:56.883066 24364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:56.884413 24364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:56.897099 24369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:56.897131 24370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:56.897089 24372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:56.897845 24364 server_base.cc:1034] running on GCE node
I20260928 15:56:56.898429 24364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:56.899349 24364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:56.900556 24364 hybrid_clock.cc:648] HybridClock initialized: now 1790611016900490 us; error 42 us; skew 500 ppm
I20260928 15:56:56.900848 24364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:56 dist-test-slave-9cqd krb5kdc[24109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:56.904871 24364 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:56:56.912384 24364 webserver.cc:460] Webserver started at http://127.22.125.194:37879/ using document root <none> and password file <none>
I20260928 15:56:56.913735 24364 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:56.913880 24364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:56.914358 24364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:56.919359 24364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "0c69f561315d470181f7017a13b13a79"
format_stamp: "Formatted at 2026-09-28 15:56:56 on dist-test-slave-9cqd"
server_key: "6039bae7523bfe583f6640d4491f4955"
server_key_iv: "0a48102c2ff700ddaf0bceb87ab7bbb0"
server_key_version: "encryptionkey@0"
I20260928 15:56:56.928215 24364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "0c69f561315d470181f7017a13b13a79"
format_stamp: "Formatted at 2026-09-28 15:56:56 on dist-test-slave-9cqd"
server_key: "6039bae7523bfe583f6640d4491f4955"
server_key_iv: "0a48102c2ff700ddaf0bceb87ab7bbb0"
server_key_version: "encryptionkey@0"
I20260928 15:56:56.935309 24364 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260928 15:56:56.940362 24379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:56.942363 24364 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 15:56:56.942536 24364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "0c69f561315d470181f7017a13b13a79"
format_stamp: "Formatted at 2026-09-28 15:56:56 on dist-test-slave-9cqd"
server_key: "6039bae7523bfe583f6640d4491f4955"
server_key_iv: "0a48102c2ff700ddaf0bceb87ab7bbb0"
server_key_version: "encryptionkey@0"
I20260928 15:56:56.942749 24364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:56.963179 24364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:56.967641 24364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:56.967931 24364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:56.970738 24364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:56.970866 24364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:56.971012 24364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:56.971069 24364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:56:56 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:56.994622 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:56.972160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:34337 (local address 127.22.125.254:36909)
0928 15:56:56.972380 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:56.972388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:56.972764 (+ 376us) server_negotiation.cc:407] Connection header received
0928 15:56:56.975048 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:56.975055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:56.975152 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:56.975313 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:56.977359 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.978732 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.981325 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.981763 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.981902 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:56.986918 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:56.986923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:56.986926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:56.986984 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:56.990867 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:56.991865 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:56.991871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:56.991875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:56.992085 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:56.992754 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:56.992762 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:56.992767 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:56.993143 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:56.993316 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:56.994051 (+ 735us) server_negotiation.cc:299] Negotiation successful
0928 15:56:56.994282 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 15:56:56.995653 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:56.971653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36909 (local address 127.22.125.194:34337)
0928 15:56:56.972582 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:56.972634 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:56.974764 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:56.975657 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:56.975668 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:56.976555 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:56.977183 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:56.977200 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.978946 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.978952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:56.981174 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:56.981186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:56.981991 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:56.981999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:56.982225 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:56.982933 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:56.982978 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:56.986574 (+ 3596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:56.991042 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:56.991057 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:56.991074 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:56.991721 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:56.992239 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:56.992247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:56.992253 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:56.992616 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:56.993291 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:56.993299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:56.993656 (+ 357us) client_negotiation.cc:769] Sending connection context
0928 15:56:56.993944 (+ 288us) client_negotiation.cc:240] Negotiation successful
0928 15:56:56.994444 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"spinlock_wait_cycles":18176,"thread_start_us":514,"threads_started":1}
I20260928 15:56:57.032498 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.010507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36909 (local address 127.22.125.194:48569)
0928 15:56:57.011621 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:57.011647 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:57.011879 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:57.012554 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:57.012561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:57.013218 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:57.013829 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.013852 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.015562 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.015570 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.017831 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.017852 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.019290 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.019298 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.019567 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.022267 (+ 2700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:57.022311 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:57.023602 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:57.028763 (+ 5161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:57.028772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:57.028779 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:57.029422 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:57.029973 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:57.029981 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:57.029988 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:57.030195 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:57.031259 (+ 1064us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:57.031264 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:57.031466 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 15:56:57.031623 (+ 157us) client_negotiation.cc:240] Negotiation successful
0928 15:56:57.032052 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"thread_start_us":745,"threads_started":1}
I20260928 15:56:57.033061 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.010502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:48569 (local address 127.22.125.254:36909)
0928 15:56:57.010714 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:57.010723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:57.011737 (+ 1014us) server_negotiation.cc:407] Connection header received
0928 15:56:57.012109 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:57.012118 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:57.012223 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:57.012310 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:57.014049 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.015344 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.018060 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.018824 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.018986 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.023800 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:57.023809 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:57.023817 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:57.023866 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:57.028557 (+ 4691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.029634 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.029642 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.029649 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.029811 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.030381 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.030387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.030390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.030965 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:57.031150 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:57.032329 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0928 15:56:57.032646 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 15:56:57.034799 24364 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:35805
I20260928 15:56:57.034837 24498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:35805 every 8 connection(s)
I20260928 15:56:57.036119 24364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:56:57.038302 24499 heartbeater.cc:347] Connected to a master server at 127.22.125.254:36909
I20260928 15:56:57.038486 24499 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:57.039064 24499 heartbeater.cc:511] Master 127.22.125.254:36909 requested a full tablet report, sending...
I20260928 15:56:57.040279 24154 ts_manager.cc:194] Registered new tserver with Master: 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805)
I20260928 15:56:57.041044 24154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:34337
I20260928 15:56:57.044289 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 24364
I20260928 15:56:57.044454 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:56:57.044857 23031 external_mini_cluster.cc:1428] Setting key 4a1390cd7811d472154c6afe6335637f
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:56:57.083248 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36909
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:56:57.392802 24503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:56:57.393056 24503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:56:57.393218 24503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:56:57.400437 24503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:56:57.400573 24503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:56:57.400743 24503 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:56:57.400868 24503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:56:57.410149 24503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:36909
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:56:57.411355 24503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:56:57.413178 24503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:56:57.425901 24509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:57.425854 24511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:56:57.425931 24508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:56:57.426867 24503 server_base.cc:1034] running on GCE node
I20260928 15:56:57.427492 24503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:56:57.428438 24503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:56:57.429684 24503 hybrid_clock.cc:648] HybridClock initialized: now 1790611017429608 us; error 51 us; skew 500 ppm
I20260928 15:56:57.429984 24503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:56:57 dist-test-slave-9cqd krb5kdc[24109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:56:57.433120 24503 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:56:57.434427 24503 webserver.cc:460] Webserver started at http://127.22.125.195:35161/ using document root <none> and password file <none>
I20260928 15:56:57.435089 24503 fs_manager.cc:359] Metadata directory not provided
I20260928 15:56:57.435228 24503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:56:57.435559 24503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:56:57.438484 24503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "ebff7dba3f264f59ac956c0e291fdc25"
format_stamp: "Formatted at 2026-09-28 15:56:57 on dist-test-slave-9cqd"
server_key: "3324d7e36b8617110b5af45d0adc978b"
server_key_iv: "ff9597c7ca59431175529b53efdf7b9a"
server_key_version: "encryptionkey@0"
I20260928 15:56:57.439278 24503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "ebff7dba3f264f59ac956c0e291fdc25"
format_stamp: "Formatted at 2026-09-28 15:56:57 on dist-test-slave-9cqd"
server_key: "3324d7e36b8617110b5af45d0adc978b"
server_key_iv: "ff9597c7ca59431175529b53efdf7b9a"
server_key_version: "encryptionkey@0"
I20260928 15:56:57.444192 24503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 15:56:57.447579 24518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:57.448951 24503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:56:57.449126 24503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "ebff7dba3f264f59ac956c0e291fdc25"
format_stamp: "Formatted at 2026-09-28 15:56:57 on dist-test-slave-9cqd"
server_key: "3324d7e36b8617110b5af45d0adc978b"
server_key_iv: "ff9597c7ca59431175529b53efdf7b9a"
server_key_version: "encryptionkey@0"
I20260928 15:56:57.449352 24503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:56:57.468601 24503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:56:57.473425 24503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:56:57.473714 24503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:56:57.476375 24503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:56:57.476543 24503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:56:57.476687 24503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:56:57.476775 24503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:56:57 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611017, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:57.504429 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.476981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:54295 (local address 127.22.125.254:36909)
0928 15:56:57.477212 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:57.477218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:57.478737 (+ 1519us) server_negotiation.cc:407] Connection header received
0928 15:56:57.481790 (+ 3053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:57.481796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:57.481883 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:57.482000 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:57.484671 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.485636 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.488882 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.489300 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.489402 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.495023 (+ 5621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:57.495031 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:57.495037 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:57.495082 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:57.499021 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.500204 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.500212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.500217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.500383 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.501156 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.501163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.501166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.501591 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:57.501766 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:57.503797 (+ 2031us) server_negotiation.cc:299] Negotiation successful
0928 15:56:57.504071 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 15:56:57.505110 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.477599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36909 (local address 127.22.125.195:54295)
0928 15:56:57.478552 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:57.478602 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:57.481445 (+ 2843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:57.482326 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:57.482342 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:57.483327 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:57.484450 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.484488 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.485817 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.485825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.488712 (+ 2887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.488734 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.489582 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.489592 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.489977 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.490807 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:57.490859 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:57.494648 (+ 3789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:57.499203 (+ 4555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:57.499214 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:57.499231 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:57.499978 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:57.500553 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:57.500559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:57.500569 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:57.500887 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:57.501849 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:57.501863 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:57.502406 (+ 543us) client_negotiation.cc:769] Sending connection context
0928 15:56:57.502864 (+ 458us) client_negotiation.cc:240] Negotiation successful
0928 15:56:57.503743 (+ 879us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":361,"threads_started":1}
I20260928 15:56:57.509927 24360 heartbeater.cc:503] Master 127.22.125.254:36909 was elected leader, sending a full tablet report...
I20260928 15:56:57.527778 24503 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:35987
I20260928 15:56:57.527866 24637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:35987 every 8 connection(s)
I20260928 15:56:57.529484 24503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:56:57.529512 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.511526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36909 (local address 127.22.125.195:35699)
0928 15:56:57.512460 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:57.512482 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:57.512628 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:57.513111 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:57.513120 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:57.513937 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:57.514535 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.514558 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.516171 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.516180 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.518685 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.518705 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.519514 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.519520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.519753 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.520544 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:57.520576 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:57.522152 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:57.526440 (+ 4288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:57.526447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:57.526453 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:57.527085 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:57.527616 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:57.527623 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:57.527629 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:57.527809 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:57.528495 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:57.528498 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:57.528656 (+ 158us) client_negotiation.cc:769] Sending connection context
0928 15:56:57.528826 (+ 170us) client_negotiation.cc:240] Negotiation successful
0928 15:56:57.529129 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":697,"threads_started":1}
I20260928 15:56:57.529623 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.512006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:35699 (local address 127.22.125.254:36909)
0928 15:56:57.512229 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:57.512237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:57.512560 (+ 323us) server_negotiation.cc:407] Connection header received
0928 15:56:57.512771 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:57.512804 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:57.512897 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:57.513032 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:57.514711 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.515957 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.518895 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.519315 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.519419 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.522414 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:57.522420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:57.522425 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:57.522464 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:57.526280 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.527250 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.527259 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.527265 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.527473 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.528015 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.528021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.528026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.528364 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:57.528521 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:57.528943 (+ 422us) server_negotiation.cc:299] Negotiation successful
0928 15:56:57.529161 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 15:56:57.531538 24638 heartbeater.cc:347] Connected to a master server at 127.22.125.254:36909
I20260928 15:56:57.532609 24638 heartbeater.cc:464] Registering TS with master...
I20260928 15:56:57.533239 24638 heartbeater.cc:511] Master 127.22.125.254:36909 requested a full tablet report, sending...
I20260928 15:56:57.534502 24154 ts_manager.cc:194] Registered new tserver with Master: ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987)
I20260928 15:56:57.535216 24154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:54295
I20260928 15:56:57.539342 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 24503
I20260928 15:56:57.539546 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:56:57.539987 23031 external_mini_cluster.cc:1428] Setting key 190efdc941ac3d3b2170de7720f6bda1
I20260928 15:56:57.542683 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 15:56:57 dist-test-slave-9cqd krb5kdc[24109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 15:56:57 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:57.587427 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.573879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.22.125.254:36909)
0928 15:56:57.574072 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:57.574076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:57.574393 (+ 317us) server_negotiation.cc:407] Connection header received
0928 15:56:57.574553 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:57.574557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:57.574622 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:57.574716 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:57.576118 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.576984 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.578270 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.578679 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.578772 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.582440 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:57.582445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:57.582448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:57.582482 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:57.584873 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.585434 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.585438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.585440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.585562 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:57.585972 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:57.585978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:57.585981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:57.586273 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:57.586531 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:57.586853 (+ 322us) server_negotiation.cc:299] Negotiation successful
0928 15:56:57.587075 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 15:56:57.591353 24154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:56:57.598284 24154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:56:57.623190 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.610751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.22.125.195:35987)
0928 15:56:57.612321 (+ 1570us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:57.612332 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:57.612375 (+ 43us) server_negotiation.cc:407] Connection header received
0928 15:56:57.612507 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:57.612519 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:57.612826 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:57.615511 (+ 2685us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:57.615823 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.617600 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.620654 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.622085 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.622333 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.622573 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:57.622662 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 15:56:57.622829 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1377,"thread_start_us":1182,"threads_started":1}
I20260928 15:56:57.623225 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.610298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:35987 (local address 127.0.0.1:49814)
0928 15:56:57.611175 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:57.611220 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:57.611416 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:57.613646 (+ 2230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:57.613657 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:57.613700 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:57.614950 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.614968 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.617770 (+ 2802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.617776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.620469 (+ 2693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.620484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.622355 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.622362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.622518 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.622542 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 15:56:57.622609 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 15:56:57.622819 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":10,"thread_start_us":682,"threads_started":1}
I20260928 15:56:57.623227 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.610970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.22.125.194:35805)
0928 15:56:57.611768 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:57.611778 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:57.614598 (+ 2820us) server_negotiation.cc:407] Connection header received
0928 15:56:57.614728 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:57.614739 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:57.615027 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:57.615250 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:57.616221 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.617650 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.620609 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.621663 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.621840 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.622021 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:57.622786 (+ 765us) server_negotiation.cc:299] Negotiation successful
0928 15:56:57.622951 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":522,"threads_started":1}
I20260928 15:56:57.623986 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.610379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:35805 (local address 127.0.0.1:58856)
0928 15:56:57.614062 (+ 3683us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:57.614085 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:57.614225 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:57.615473 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:57.615477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:57.615501 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:57.615958 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.615971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.617789 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.617795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.620462 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.620474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.622148 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.622154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.622443 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.622501 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 15:56:57.622609 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 15:56:57.623716 (+ 1107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3512,"mutex_wait_us":1100,"thread_start_us":2900,"threads_started":1}
I20260928 15:56:57.624565 24655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.611787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:37973 (local address 127.0.0.1:38402)
0928 15:56:57.615645 (+ 3858us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:57.615669 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:57.615805 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:57.616715 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:57.616720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:57.616743 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:57.617056 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.617064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.619048 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.619062 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.621251 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:57.621265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.623734 (+ 2469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.623741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:57.623934 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.623969 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 15:56:57.624080 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 15:56:57.624192 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3673,"mutex_wait_us":31,"thread_start_us":3462,"threads_started":1}
I20260928 15:56:57.626703 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:57.612212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.22.125.193:37973)
0928 15:56:57.616300 (+ 4088us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:57.616305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:57.616326 (+ 21us) server_negotiation.cc:407] Connection header received
0928 15:56:57.616402 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:57.616407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:57.616546 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:57.616728 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:57.617289 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.618873 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.621436 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:57.622970 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:57.623139 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:57.624592 (+ 1453us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:57.624666 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 15:56:57.624837 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3942,"thread_start_us":3799,"threads_started":1}
I20260928 15:56:57.627065 24568 tablet_service.cc:1467] Processing CreateTablet for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 (DEFAULT_TABLE table=test-table [id=cb91bad1c0254e71b58d2b6bfca8cfa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:57.627058 24429 tablet_service.cc:1467] Processing CreateTablet for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 (DEFAULT_TABLE table=test-table [id=cb91bad1c0254e71b58d2b6bfca8cfa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:57.628388 24429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:57.628464 24568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:57.630105 24290 tablet_service.cc:1467] Processing CreateTablet for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 (DEFAULT_TABLE table=test-table [id=cb91bad1c0254e71b58d2b6bfca8cfa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:57.631345 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:57.640628 24657 tablet_bootstrap.cc:492] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Bootstrap starting.
I20260928 15:56:57.642715 24658 tablet_bootstrap.cc:492] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Bootstrap starting.
I20260928 15:56:57.643793 24657 tablet_bootstrap.cc:654] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:57.644923 24659 tablet_bootstrap.cc:492] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Bootstrap starting.
I20260928 15:56:57.645164 24657 log.cc:824] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:57.646857 24658 tablet_bootstrap.cc:654] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:57.647993 24659 tablet_bootstrap.cc:654] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:57.648589 24658 log.cc:824] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:57.649273 24659 log.cc:824] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Log is configured to *not* fsync() on all Append() calls
I20260928 15:56:57.651747 24659 tablet_bootstrap.cc:492] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: No bootstrap required, opened a new log
I20260928 15:56:57.651889 24658 tablet_bootstrap.cc:492] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: No bootstrap required, opened a new log
I20260928 15:56:57.651889 24657 tablet_bootstrap.cc:492] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: No bootstrap required, opened a new log
I20260928 15:56:57.652074 24659 ts_tablet_manager.cc:1397] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260928 15:56:57.652184 24657 ts_tablet_manager.cc:1397] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260928 15:56:57.652184 24658 ts_tablet_manager.cc:1397] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260928 15:56:57.659888 24659 raft_consensus.cc:348] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:57.660329 24659 raft_consensus.cc:374] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:57.660439 24659 raft_consensus.cc:729] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e26707648940a2a35fd6c0bf5eba66, State: Initialized, Role: FOLLOWER
I20260928 15:56:57.661113 24659 consensus_queue.cc:260] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:57.663064 24659 ts_tablet_manager.cc:1428] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260928 15:56:57.662894 24657 raft_consensus.cc:348] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:57.662892 24658 raft_consensus.cc:348] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:57.663293 24658 raft_consensus.cc:374] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:57.663338 24657 raft_consensus.cc:374] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:57.663388 24658 raft_consensus.cc:729] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebff7dba3f264f59ac956c0e291fdc25, State: Initialized, Role: FOLLOWER
I20260928 15:56:57.663429 24657 raft_consensus.cc:729] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c69f561315d470181f7017a13b13a79, State: Initialized, Role: FOLLOWER
I20260928 15:56:57.664016 24657 consensus_queue.cc:260] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:57.663978 24658 consensus_queue.cc:260] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:57.665082 24499 heartbeater.cc:503] Master 127.22.125.254:36909 was elected leader, sending a full tablet report...
I20260928 15:56:57.665629 24638 heartbeater.cc:503] Master 127.22.125.254:36909 was elected leader, sending a full tablet report...
I20260928 15:56:57.665689 24657 ts_tablet_manager.cc:1428] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260928 15:56:57.667758 24658 ts_tablet_manager.cc:1428] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260928 15:56:57.751577 24361 tablet.cc:2369] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:56:57.781883 24639 tablet.cc:2369] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:56:57.788200 24500 tablet.cc:2369] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:56:58.006074 24663 raft_consensus.cc:482] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:56:58.006398 24663 raft_consensus.cc:504] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:58.008332 24663 leader_election.cc:288] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987), 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805)
I20260928 15:56:58.015228 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.009020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:45825 (local address 127.22.125.194:35805)
0928 15:56:58.009210 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.009219 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.009636 (+ 417us) server_negotiation.cc:407] Connection header received
0928 15:56:58.009822 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.009827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.009886 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.009993 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:58.010597 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.011323 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.013277 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.014175 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.014355 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.014443 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.014810 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.014952 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 15:56:58.015239 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.008718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:57467 (local address 127.22.125.195:35987)
0928 15:56:58.008926 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.008934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.009710 (+ 776us) server_negotiation.cc:407] Connection header received
0928 15:56:58.009923 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.009927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.009986 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.010083 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:58.010699 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.011396 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.013285 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.014186 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.014403 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.014484 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.014836 (+ 352us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.014964 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 15:56:58.015377 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.008649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:35805 (local address 127.22.125.193:45825)
0928 15:56:58.009515 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.009531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.009660 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.010008 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.010013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.010049 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:58.010440 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.010450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.011460 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.011466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.013121 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.013134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.014424 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.014430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.014631 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.014677 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.014784 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.014945 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":70,"thread_start_us":658,"threads_started":1}
I20260928 15:56:58.015812 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.008718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:35987 (local address 127.22.125.193:57467)
0928 15:56:58.009614 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.009633 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.009769 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.010102 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.010106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.010146 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:58.010545 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.010554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.011538 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.011544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.013137 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.013150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.014471 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.014477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.014681 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.014704 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.014810 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.014965 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":89,"thread_start_us":416,"threads_started":1}
I20260928 15:56:58.016558 24453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995ec1f5e8ab4ee0bfbc0a7394dcc9a7" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c69f561315d470181f7017a13b13a79" is_pre_election: true
I20260928 15:56:58.016775 24593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995ec1f5e8ab4ee0bfbc0a7394dcc9a7" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebff7dba3f264f59ac956c0e291fdc25" is_pre_election: true
I20260928 15:56:58.017181 24593 raft_consensus.cc:2454] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 0.
I20260928 15:56:58.017174 24453 raft_consensus.cc:2454] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 0.
I20260928 15:56:58.017809 24243 leader_election.cc:302] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2e26707648940a2a35fd6c0bf5eba66, ebff7dba3f264f59ac956c0e291fdc25; no voters:
I20260928 15:56:58.018141 24663 raft_consensus.cc:2781] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:56:58.018306 24663 raft_consensus.cc:482] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:56:58.018437 24663 raft_consensus.cc:3037] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.020774 24663 raft_consensus.cc:504] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:58.021417 24663 leader_election.cc:288] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 election: Requested vote from peers ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987), 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805)
I20260928 15:56:58.021783 24593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995ec1f5e8ab4ee0bfbc0a7394dcc9a7" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebff7dba3f264f59ac956c0e291fdc25"
I20260928 15:56:58.022001 24593 raft_consensus.cc:3037] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.021940 24453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "995ec1f5e8ab4ee0bfbc0a7394dcc9a7" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c69f561315d470181f7017a13b13a79"
I20260928 15:56:58.022156 24453 raft_consensus.cc:3037] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.024308 24593 raft_consensus.cc:2454] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 1.
I20260928 15:56:58.024497 24453 raft_consensus.cc:2454] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 1.
I20260928 15:56:58.024708 24243 leader_election.cc:302] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2e26707648940a2a35fd6c0bf5eba66, ebff7dba3f264f59ac956c0e291fdc25; no voters:
I20260928 15:56:58.024996 24663 raft_consensus.cc:2781] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:56:58.025416 24663 raft_consensus.cc:686] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 LEADER]: Becoming Leader. State: Replica: a2e26707648940a2a35fd6c0bf5eba66, State: Running, Role: LEADER
I20260928 15:56:58.025877 24663 consensus_queue.cc:237] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } }
I20260928 15:56:58.030356 24153 catalog_manager.cc:5495] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193). New cstate: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } health_report { overall_health: HEALTHY } } }
I20260928 15:56:58.089095 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.083634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38410 (local address 127.22.125.193:37973)
0928 15:56:58.083835 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.083840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.083984 (+ 144us) server_negotiation.cc:407] Connection header received
0928 15:56:58.084369 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.084373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.084452 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.084574 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:56:58.084893 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.085615 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.086930 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.087369 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.087462 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.087970 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.088599 (+ 629us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.088713 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.088829 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.088894 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 15:56:58.097563 24453 raft_consensus.cc:1261] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Refusing update from remote peer a2e26707648940a2a35fd6c0bf5eba66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:56:58.098606 24668 consensus_queue.cc:1035] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:56:58.099412 24593 raft_consensus.cc:1261] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Refusing update from remote peer a2e26707648940a2a35fd6c0bf5eba66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:56:58.100242 24668 consensus_queue.cc:1035] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:56:58.111752 24672 mvcc.cc:205] Tried to move back new op lower bound from 7334342730116583424 to 7334342729837711360. Current Snapshot: MvccSnapshot[applied={T|T < 7334342730116583424}]
I20260928 15:56:58.138578 24153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:56:58.138914 24153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:56:58.142800 24153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6: Sending DeleteTablet for 3 replicas of tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7
I20260928 15:56:58.144330 24568 tablet_service.cc:1514] Processing DeleteTablet for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:56:58 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:49814
I20260928 15:56:58.144423 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:56:58 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:38402
I20260928 15:56:58.144925 24429 tablet_service.cc:1514] Processing DeleteTablet for tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:56:58 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:58856
I20260928 15:56:58.145606 24153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:56:58.146340 24153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:56:58.149788 24683 tablet_replica.cc:330] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: stopping tablet replica
I20260928 15:56:58.150593 24683 raft_consensus.cc:2229] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:56:58.151113 24683 raft_consensus.cc:2258] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:56:58.157629 24289 tablet_service.cc:1467] Processing CreateTablet for tablet d1ecb1b9cd1740939d604189fa5fef00 (DEFAULT_TABLE table=test-table [id=8406a5403c254f3bad3e135d959559e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:58.158352 24289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ecb1b9cd1740939d604189fa5fef00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:58.160928 24567 tablet_service.cc:1467] Processing CreateTablet for tablet d1ecb1b9cd1740939d604189fa5fef00 (DEFAULT_TABLE table=test-table [id=8406a5403c254f3bad3e135d959559e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:58.161037 24429 tablet_service.cc:1467] Processing CreateTablet for tablet d1ecb1b9cd1740939d604189fa5fef00 (DEFAULT_TABLE table=test-table [id=8406a5403c254f3bad3e135d959559e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:56:58.161414 24429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ecb1b9cd1740939d604189fa5fef00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:58.161430 24567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ecb1b9cd1740939d604189fa5fef00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:58.163540 24683 ts_tablet_manager.cc:1905] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:56:58.168087 24682 tablet_replica.cc:330] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: stopping tablet replica
I20260928 15:56:58.168339 24658 tablet_bootstrap.cc:492] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: Bootstrap starting.
I20260928 15:56:58.168726 24682 raft_consensus.cc:2229] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:56:58.169639 24682 raft_consensus.cc:2258] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:56:58.170536 24658 tablet_bootstrap.cc:654] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:58.173702 24683 ts_tablet_manager.cc:1918] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:56:58.174993 24683 log.cc:1196] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/995ec1f5e8ab4ee0bfbc0a7394dcc9a7
I20260928 15:56:58.175283 24681 tablet_replica.cc:330] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: stopping tablet replica
I20260928 15:56:58.175624 24683 ts_tablet_manager.cc:1939] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P 0c69f561315d470181f7017a13b13a79: Deleting consensus metadata
I20260928 15:56:58.175843 24681 raft_consensus.cc:2229] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:56:58.176478 24681 raft_consensus.cc:2258] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:56:58.177356 24684 tablet_bootstrap.cc:492] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: Bootstrap starting.
I20260928 15:56:58.177610 24682 ts_tablet_manager.cc:1905] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:56:58.177639 24138 catalog_manager.cc:4841] TS 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805): tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 (table test-table [id=cb91bad1c0254e71b58d2b6bfca8cfa0]) successfully deleted
I20260928 15:56:58.179735 24684 tablet_bootstrap.cc:654] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:58.180344 24658 tablet_bootstrap.cc:492] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: No bootstrap required, opened a new log
I20260928 15:56:58.180495 24658 ts_tablet_manager.cc:1397] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260928 15:56:58.181367 24681 ts_tablet_manager.cc:1905] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:56:58.181329 24658 raft_consensus.cc:348] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.181566 24658 raft_consensus.cc:374] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:58.181668 24658 raft_consensus.cc:729] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebff7dba3f264f59ac956c0e291fdc25, State: Initialized, Role: FOLLOWER
I20260928 15:56:58.182034 24658 consensus_queue.cc:260] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.184980 24658 ts_tablet_manager.cc:1428] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260928 15:56:58.188246 24685 tablet_bootstrap.cc:492] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: Bootstrap starting.
I20260928 15:56:58.188405 24681 ts_tablet_manager.cc:1918] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:56:58.188567 24681 log.cc:1196] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/995ec1f5e8ab4ee0bfbc0a7394dcc9a7
I20260928 15:56:58.189095 24681 ts_tablet_manager.cc:1939] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P ebff7dba3f264f59ac956c0e291fdc25: Deleting consensus metadata
I20260928 15:56:58.190055 24685 tablet_bootstrap.cc:654] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:58.190935 24140 catalog_manager.cc:4841] TS ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987): tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 (table test-table [id=cb91bad1c0254e71b58d2b6bfca8cfa0]) successfully deleted
I20260928 15:56:58.192324 24682 ts_tablet_manager.cc:1918] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:56:58.192582 24682 log.cc:1196] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/995ec1f5e8ab4ee0bfbc0a7394dcc9a7
I20260928 15:56:58.192619 24685 tablet_bootstrap.cc:492] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: No bootstrap required, opened a new log
I20260928 15:56:58.192754 24685 ts_tablet_manager.cc:1397] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260928 15:56:58.193307 24684 tablet_bootstrap.cc:492] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: No bootstrap required, opened a new log
I20260928 15:56:58.193305 24682 ts_tablet_manager.cc:1939] T 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 P a2e26707648940a2a35fd6c0bf5eba66: Deleting consensus metadata
I20260928 15:56:58.193496 24684 ts_tablet_manager.cc:1397] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.005s
I20260928 15:56:58.193617 24685 raft_consensus.cc:348] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.193876 24685 raft_consensus.cc:374] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:58.193971 24685 raft_consensus.cc:729] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c69f561315d470181f7017a13b13a79, State: Initialized, Role: FOLLOWER
I20260928 15:56:58.194308 24685 consensus_queue.cc:260] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.194478 24684 raft_consensus.cc:348] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.194854 24684 raft_consensus.cc:374] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:58.195017 24685 ts_tablet_manager.cc:1428] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:56:58.195022 24684 raft_consensus.cc:729] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e26707648940a2a35fd6c0bf5eba66, State: Initialized, Role: FOLLOWER
I20260928 15:56:58.195322 24684 consensus_queue.cc:260] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.196488 24140 catalog_manager.cc:4841] TS a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193:37973): tablet 995ec1f5e8ab4ee0bfbc0a7394dcc9a7 (table test-table [id=cb91bad1c0254e71b58d2b6bfca8cfa0]) successfully deleted
I20260928 15:56:58.197080 24684 ts_tablet_manager.cc:1428] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260928 15:56:58.383741 24665 raft_consensus.cc:482] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:56:58.384011 24665 raft_consensus.cc:504] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.386085 24665 leader_election.cc:288] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193:37973), 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805)
I20260928 15:56:58.394606 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.387207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:41583 (local address 127.22.125.194:35805)
0928 15:56:58.387405 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.387414 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.388291 (+ 877us) server_negotiation.cc:407] Connection header received
0928 15:56:58.388574 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.388580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.388650 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.388712 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:58.389562 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.390486 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.392495 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.393570 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.393735 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.393809 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.394154 (+ 345us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.394293 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 15:56:58.394878 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.386767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:39189 (local address 127.22.125.193:37973)
0928 15:56:58.387015 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.387025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.387987 (+ 962us) server_negotiation.cc:407] Connection header received
0928 15:56:58.388330 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.388335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.388414 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.388510 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:58.389022 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.390096 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.392608 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.393871 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.394040 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.394126 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.394428 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.394573 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 15:56:58.394894 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.386316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:37973 (local address 127.22.125.195:39189)
0928 15:56:58.387864 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.387887 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.388164 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.388528 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.388532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.388559 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:58.388863 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.388871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.390249 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.390254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.392427 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.392442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.394096 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.394105 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.394269 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.394299 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.394388 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.394534 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1362,"thread_start_us":1496,"threads_started":1}
I20260928 15:56:58.395694 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.386316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:35805 (local address 127.22.125.195:41583)
0928 15:56:58.388167 (+ 1851us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.388194 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.388408 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.388954 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.388960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.388989 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:58.389372 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.389387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.390609 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.390615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.392308 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.392321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.393762 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.393768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.393927 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.393983 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.395197 (+ 1214us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.395330 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1700,"mutex_wait_us":113,"thread_start_us":1041,"threads_started":1}
I20260928 15:56:58.395761 24313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ecb1b9cd1740939d604189fa5fef00" candidate_uuid: "ebff7dba3f264f59ac956c0e291fdc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2e26707648940a2a35fd6c0bf5eba66" is_pre_election: true
I20260928 15:56:58.396307 24313 raft_consensus.cc:2454] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebff7dba3f264f59ac956c0e291fdc25 in term 0.
I20260928 15:56:58.396457 24453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ecb1b9cd1740939d604189fa5fef00" candidate_uuid: "ebff7dba3f264f59ac956c0e291fdc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c69f561315d470181f7017a13b13a79" is_pre_election: true
I20260928 15:56:58.396701 24453 raft_consensus.cc:2454] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebff7dba3f264f59ac956c0e291fdc25 in term 0.
I20260928 15:56:58.397010 24521 leader_election.cc:302] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2e26707648940a2a35fd6c0bf5eba66, ebff7dba3f264f59ac956c0e291fdc25; no voters:
I20260928 15:56:58.397426 24665 raft_consensus.cc:2781] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:56:58.397549 24665 raft_consensus.cc:482] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:56:58.397645 24665 raft_consensus.cc:3037] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.399865 24665 raft_consensus.cc:504] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.400558 24665 leader_election.cc:288] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [CANDIDATE]: Term 1 election: Requested vote from peers a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193:37973), 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805)
I20260928 15:56:58.401038 24313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ecb1b9cd1740939d604189fa5fef00" candidate_uuid: "ebff7dba3f264f59ac956c0e291fdc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2e26707648940a2a35fd6c0bf5eba66"
I20260928 15:56:58.401099 24453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ecb1b9cd1740939d604189fa5fef00" candidate_uuid: "ebff7dba3f264f59ac956c0e291fdc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c69f561315d470181f7017a13b13a79"
I20260928 15:56:58.401237 24313 raft_consensus.cc:3037] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.401350 24453 raft_consensus.cc:3037] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.403652 24313 raft_consensus.cc:2454] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebff7dba3f264f59ac956c0e291fdc25 in term 1.
I20260928 15:56:58.404371 24521 leader_election.cc:302] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2e26707648940a2a35fd6c0bf5eba66, ebff7dba3f264f59ac956c0e291fdc25; no voters:
I20260928 15:56:58.404464 24453 raft_consensus.cc:2454] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebff7dba3f264f59ac956c0e291fdc25 in term 1.
I20260928 15:56:58.404639 24665 raft_consensus.cc:2781] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:56:58.405196 24665 raft_consensus.cc:686] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 LEADER]: Becoming Leader. State: Replica: ebff7dba3f264f59ac956c0e291fdc25, State: Running, Role: LEADER
I20260928 15:56:58.405612 24665 consensus_queue.cc:237] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.409375 24153 catalog_manager.cc:5495] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195). New cstate: current_term: 1 leader_uuid: "ebff7dba3f264f59ac956c0e291fdc25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } health_report { overall_health: HEALTHY } } }
Sep 28 15:56:58 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:56:58.498027 24698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.474450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36909 (local address 127.0.0.1:34126)
0928 15:56:58.475140 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.475166 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.475361 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.476599 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.476604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.477523 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:56:58.477796 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.477812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.479497 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.479501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.480561 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.480573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.481561 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.481565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.481755 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.488412 (+ 6657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:56:58.488433 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:56:58.491028 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:56:58.494985 (+ 3957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:58.494996 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:58.495002 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:58.495624 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:58.496031 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:56:58.496034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:56:58.496036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:56:58.496237 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:56:58.496918 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:56:58.496927 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:56:58.497124 (+ 197us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.497298 (+ 174us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.497602 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":39,"thread_start_us":383,"threads_started":1}
I20260928 15:56:58.499074 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.475205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.22.125.254:36909)
0928 15:56:58.476203 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.476209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.476234 (+ 25us) server_negotiation.cc:407] Connection header received
0928 15:56:58.476292 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.476297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.476392 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.476515 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:58.478042 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.479367 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.480709 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.481402 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.481548 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.491296 (+ 9748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:58.491304 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:58.491309 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:58.491349 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:58.494850 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.495757 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.495763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.495767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.495915 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.496368 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.496374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.496378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.496762 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:58.496917 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.497435 (+ 518us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.497849 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":856,"thread_start_us":713,"threads_started":1}
I20260928 15:56:58.510013 24153 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:34126:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 15:56:58.511415 24153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:56:58.519052 24289 tablet_service.cc:1467] Processing CreateTablet for tablet 38fb4ec93b5b4df3a6b7c5b6ae0366af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0f209ca67565495c9b6d3a99952224f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:56:58.519658 24289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38fb4ec93b5b4df3a6b7c5b6ae0366af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:58.519467 24429 tablet_service.cc:1467] Processing CreateTablet for tablet 38fb4ec93b5b4df3a6b7c5b6ae0366af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0f209ca67565495c9b6d3a99952224f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:56:58.520009 24429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38fb4ec93b5b4df3a6b7c5b6ae0366af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:58.520638 24568 tablet_service.cc:1467] Processing CreateTablet for tablet 38fb4ec93b5b4df3a6b7c5b6ae0366af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0f209ca67565495c9b6d3a99952224f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:56:58.527014 24568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38fb4ec93b5b4df3a6b7c5b6ae0366af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:56:58.530146 24684 tablet_bootstrap.cc:492] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: Bootstrap starting.
I20260928 15:56:58.532018 24685 tablet_bootstrap.cc:492] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: Bootstrap starting.
I20260928 15:56:58.532341 24684 tablet_bootstrap.cc:654] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:58.533978 24685 tablet_bootstrap.cc:654] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:58.544456 24684 tablet_bootstrap.cc:492] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: No bootstrap required, opened a new log
I20260928 15:56:58.544703 24684 ts_tablet_manager.cc:1397] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260928 15:56:58.545729 24684 raft_consensus.cc:348] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.546118 24684 raft_consensus.cc:374] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:58.546245 24684 raft_consensus.cc:729] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e26707648940a2a35fd6c0bf5eba66, State: Initialized, Role: FOLLOWER
I20260928 15:56:58.546583 24684 consensus_queue.cc:260] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.546864 24658 tablet_bootstrap.cc:492] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: Bootstrap starting.
I20260928 15:56:58.547149 24668 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.547549 24668 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:56:58.548628 24658 tablet_bootstrap.cc:654] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: Neither blocks nor log segments found. Creating new log.
I20260928 15:56:58.548981 24684 ts_tablet_manager.cc:1428] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 15:56:58.549906 24685 tablet_bootstrap.cc:492] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: No bootstrap required, opened a new log
I20260928 15:56:58.550082 24685 ts_tablet_manager.cc:1397] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.010s
I20260928 15:56:58.550895 24685 raft_consensus.cc:348] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.551172 24685 raft_consensus.cc:374] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:58.551275 24685 raft_consensus.cc:729] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c69f561315d470181f7017a13b13a79, State: Initialized, Role: FOLLOWER
I20260928 15:56:58.551575 24685 consensus_queue.cc:260] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.552191 24685 ts_tablet_manager.cc:1428] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260928 15:56:58.552492 24664 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.552918 24664 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:56:58.559285 24658 tablet_bootstrap.cc:492] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: No bootstrap required, opened a new log
I20260928 15:56:58.559463 24658 ts_tablet_manager.cc:1397] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260928 15:56:58.560323 24658 raft_consensus.cc:348] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.560561 24658 raft_consensus.cc:374] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:56:58.560643 24658 raft_consensus.cc:729] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebff7dba3f264f59ac956c0e291fdc25, State: Initialized, Role: FOLLOWER
I20260928 15:56:58.560889 24658 consensus_queue.cc:260] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.561357 24665 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.561636 24665 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:56:58.562217 24658 ts_tablet_manager.cc:1428] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 15:56:58.605950 24664 raft_consensus.cc:482] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:56:58.606185 24664 raft_consensus.cc:504] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.607697 24664 leader_election.cc:288] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193:37973), ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987)
I20260928 15:56:58.610812 24668 raft_consensus.cc:482] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:56:58.610991 24668 raft_consensus.cc:504] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.611657 24668 leader_election.cc:288] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805), ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987)
I20260928 15:56:58.612422 24593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fb4ec93b5b4df3a6b7c5b6ae0366af" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebff7dba3f264f59ac956c0e291fdc25" is_pre_election: true
I20260928 15:56:58.612586 24453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fb4ec93b5b4df3a6b7c5b6ae0366af" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c69f561315d470181f7017a13b13a79" is_pre_election: true
I20260928 15:56:58.612715 24593 raft_consensus.cc:2454] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 0.
I20260928 15:56:58.612861 24453 raft_consensus.cc:2454] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 0.
I20260928 15:56:58.613175 24243 leader_election.cc:302] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2e26707648940a2a35fd6c0bf5eba66, ebff7dba3f264f59ac956c0e291fdc25; no voters:
I20260928 15:56:58.613443 24668 raft_consensus.cc:2781] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:56:58.613557 24668 raft_consensus.cc:482] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:56:58.613657 24668 raft_consensus.cc:3037] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.616089 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.607998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:54249 (local address 127.22.125.193:37973)
0928 15:56:58.608220 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.608227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.608667 (+ 440us) server_negotiation.cc:407] Connection header received
0928 15:56:58.608914 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.608924 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.609037 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.609134 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:58.609990 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.611031 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.613978 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.614942 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.615111 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.615185 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.615663 (+ 478us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.615803 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 15:56:58.616122 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.607952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:37973 (local address 127.22.125.194:54249)
0928 15:56:58.608567 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.608582 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.608738 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.609333 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.609339 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.609376 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:58.609795 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.609807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.611166 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.611172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.613829 (+ 2657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.613841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.615279 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.615285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.615484 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.615517 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.615612 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.615744 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":382,"threads_started":1}
I20260928 15:56:58.616562 24668 raft_consensus.cc:504] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.617237 24668 leader_election.cc:288] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 election: Requested vote from peers 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805), ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987)
I20260928 15:56:58.617149 24313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fb4ec93b5b4df3a6b7c5b6ae0366af" candidate_uuid: "0c69f561315d470181f7017a13b13a79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2e26707648940a2a35fd6c0bf5eba66" is_pre_election: true
I20260928 15:56:58.617350 24704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.608691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:35987 (local address 127.22.125.194:43423)
0928 15:56:58.609360 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.609378 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.609553 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.609945 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.609950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.609986 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:56:58.610335 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.610345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.611860 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.611865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.614642 (+ 2777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.614656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.616583 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.616590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.616757 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.616790 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.616883 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.617007 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":288,"threads_started":1}
I20260928 15:56:58.617538 24313 raft_consensus.cc:2379] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0c69f561315d470181f7017a13b13a79 in current term 1: Already voted for candidate a2e26707648940a2a35fd6c0bf5eba66 in this term.
I20260928 15:56:58.617599 24453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fb4ec93b5b4df3a6b7c5b6ae0366af" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c69f561315d470181f7017a13b13a79"
I20260928 15:56:58.617796 24453 raft_consensus.cc:3037] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.618044 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.608837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:43423 (local address 127.22.125.195:35987)
0928 15:56:58.609607 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.609614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.609636 (+ 22us) server_negotiation.cc:407] Connection header received
0928 15:56:58.609709 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.609714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.609789 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.609893 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:56:58.610539 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.611687 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.614808 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.616144 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.616329 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.616438 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.617541 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.617714 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":446,"threads_started":1}
I20260928 15:56:58.619774 24453 raft_consensus.cc:2454] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 1.
I20260928 15:56:58.620337 24241 leader_election.cc:302] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c69f561315d470181f7017a13b13a79, a2e26707648940a2a35fd6c0bf5eba66; no voters:
I20260928 15:56:58.620492 24593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fb4ec93b5b4df3a6b7c5b6ae0366af" candidate_uuid: "0c69f561315d470181f7017a13b13a79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebff7dba3f264f59ac956c0e291fdc25" is_pre_election: true
I20260928 15:56:58.620734 24593 raft_consensus.cc:2454] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c69f561315d470181f7017a13b13a79 in term 0.
I20260928 15:56:58.620821 24591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fb4ec93b5b4df3a6b7c5b6ae0366af" candidate_uuid: "a2e26707648940a2a35fd6c0bf5eba66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebff7dba3f264f59ac956c0e291fdc25"
I20260928 15:56:58.621034 24591 raft_consensus.cc:3037] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:56:58.621055 24668 raft_consensus.cc:2781] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:56:58.621199 24382 leader_election.cc:302] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c69f561315d470181f7017a13b13a79, ebff7dba3f264f59ac956c0e291fdc25; no voters: a2e26707648940a2a35fd6c0bf5eba66
I20260928 15:56:58.621443 24668 raft_consensus.cc:686] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [term 1 LEADER]: Becoming Leader. State: Replica: a2e26707648940a2a35fd6c0bf5eba66, State: Running, Role: LEADER
I20260928 15:56:58.621742 24668 consensus_queue.cc:237] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } }
I20260928 15:56:58.621871 24664 raft_consensus.cc:2741] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260928 15:56:58.623665 24591 raft_consensus.cc:2454] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2e26707648940a2a35fd6c0bf5eba66 in term 1.
I20260928 15:56:58.624410 24153 catalog_manager.cc:5495] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193). New cstate: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } health_report { overall_health: UNKNOWN } } }
I20260928 15:56:58.624665 24706 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: TxnStatusTablet state changed. Reason: New leader a2e26707648940a2a35fd6c0bf5eba66. Latest consensus state: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.624965 24706 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:56:58.625473 24708 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 15:56:58.625758 24708 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 15:56:58.651777 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.646450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38412 (local address 127.22.125.193:37973)
0928 15:56:58.646736 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.646745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.646814 (+ 69us) server_negotiation.cc:407] Connection header received
0928 15:56:58.647135 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.647140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.647204 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.647345 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:56:58.647727 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.648649 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.649955 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.650426 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.650582 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.651016 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.651118 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.651221 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.651365 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.651446 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 15:56:58.651809 24698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.646460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:37973 (local address 127.0.0.1:38412)
0928 15:56:58.646732 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.646753 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.646901 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.647404 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.647409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.647421 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:56:58.647590 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.647598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.648779 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.648786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.649803 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.649814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.650592 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.650598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.650772 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.650825 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.651221 (+ 396us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.651245 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.651344 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.651511 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260928 15:56:58.656311 24591 raft_consensus.cc:1261] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Refusing update from remote peer a2e26707648940a2a35fd6c0bf5eba66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:56:58.656332 24453 raft_consensus.cc:1261] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Refusing update from remote peer a2e26707648940a2a35fd6c0bf5eba66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:56:58.656971 24706 consensus_queue.cc:1035] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:56:58.657276 24668 consensus_queue.cc:1035] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:56:58.665446 24664 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: TxnStatusTablet state changed. Reason: New leader a2e26707648940a2a35fd6c0bf5eba66. Latest consensus state: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.665748 24664 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:56:58.668618 24710 mvcc.cc:205] Tried to move back new op lower bound from 7334342732409421824 to 7334342732277170176. Current Snapshot: MvccSnapshot[applied={T|T < 7334342732409421824}]
I20260928 15:56:58.671613 24664 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.671882 24664 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P 0c69f561315d470181f7017a13b13a79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:56:58.673202 24665 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: TxnStatusTablet state changed. Reason: New leader a2e26707648940a2a35fd6c0bf5eba66. Latest consensus state: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.673476 24665 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:56:58.676306 24665 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.676591 24665 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P ebff7dba3f264f59ac956c0e291fdc25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:56:58.682811 24668 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.683159 24668 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:56:58.683513 24718 tablet_replica.cc:439] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2e26707648940a2a35fd6c0bf5eba66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 } } peers { permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 } } peers { permanent_uuid: "ebff7dba3f264f59ac956c0e291fdc25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35987 } } }
I20260928 15:56:58.684010 24718 tablet_replica.cc:442] T 38fb4ec93b5b4df3a6b7c5b6ae0366af P a2e26707648940a2a35fd6c0bf5eba66: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:56:58.694662 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.688304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.22.125.195:35987)
0928 15:56:58.688478 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.688484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.688804 (+ 320us) server_negotiation.cc:407] Connection header received
0928 15:56:58.688979 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.688985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.689088 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.689250 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:56:58.689625 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.690721 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.692253 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.692921 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.693039 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.693417 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.694100 (+ 683us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.694269 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.694315 (+ 46us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.694395 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 15:56:58.711848 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.705778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:48673 (local address 127.22.125.193:37973)
0928 15:56:58.706032 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.706039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.706430 (+ 391us) server_negotiation.cc:407] Connection header received
0928 15:56:58.706605 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.706609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.706674 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.706746 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:56:58.707349 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.708212 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.710217 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.710637 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.710726 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.711248 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.711354 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.711420 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.711584 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.711645 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 15:56:58.712222 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.705712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:37973 (local address 127.22.125.195:48673)
0928 15:56:58.706318 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.706343 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.706489 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.706790 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.706795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.706826 (+ 31us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:56:58.707170 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.707199 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.708343 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.708350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.710083 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.710098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.710809 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.710815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.711002 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.711071 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.711460 (+ 389us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.711473 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.711560 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.711703 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":285,"threads_started":1}
I20260928 15:56:58.722059 24453 raft_consensus.cc:1261] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Refusing update from remote peer ebff7dba3f264f59ac956c0e291fdc25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:56:58.722306 24313 raft_consensus.cc:1261] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Refusing update from remote peer ebff7dba3f264f59ac956c0e291fdc25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:56:58.723034 24665 consensus_queue.cc:1035] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c69f561315d470181f7017a13b13a79" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:56:58.723592 24691 consensus_queue.cc:1035] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2e26707648940a2a35fd6c0bf5eba66" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 37973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:56:58.762104 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.754879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:60703 (local address 127.22.125.195:35987)
0928 15:56:58.755067 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.755073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.755799 (+ 726us) server_negotiation.cc:407] Connection header received
0928 15:56:58.756033 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.756061 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.756146 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.756244 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:56:58.757073 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.757952 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.760016 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.760707 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.760816 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.761490 (+ 674us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.761598 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.761665 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.761825 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.761889 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 15:56:58.762617 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.754944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:35987 (local address 127.22.125.193:60703)
0928 15:56:58.755694 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0928 15:56:58.755714 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:56:58.755882 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:56:58.756413 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:56:58.756418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:56:58.756432 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:56:58.756867 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.756880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.758083 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.758089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.759843 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:56:58.759860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.761012 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.761018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:56:58.761235 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.761312 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:56:58.761701 (+ 389us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:56:58.761713 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 15:56:58.761799 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 15:56:58.761985 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":454,"threads_started":1}
I20260928 15:56:58.806921 24154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:56:58.807221 24154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:56:58.810721 24154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 476dcdf34e194656b1dec9e90da8b5d6: Sending DeleteTablet for 3 replicas of tablet d1ecb1b9cd1740939d604189fa5fef00
I20260928 15:56:58.811527 24289 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ecb1b9cd1740939d604189fa5fef00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:56:58 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:38402
I20260928 15:56:58.811766 24429 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ecb1b9cd1740939d604189fa5fef00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:56:58 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:58856
I20260928 15:56:58.812558 24568 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ecb1b9cd1740939d604189fa5fef00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:56:58 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:49814
I20260928 15:56:58.816495 24733 tablet_replica.cc:330] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: stopping tablet replica
I20260928 15:56:58.816702 24732 tablet_replica.cc:330] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: stopping tablet replica
I20260928 15:56:58.816738 24733 raft_consensus.cc:2229] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:56:58.816903 24732 raft_consensus.cc:2229] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:56:58.817037 24733 raft_consensus.cc:2258] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:56:58.817168 24732 raft_consensus.cc:2258] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:56:58.818323 24732 ts_tablet_manager.cc:1905] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:56:58.824179 24734 tablet_replica.cc:330] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: stopping tablet replica
I20260928 15:56:58.824460 24734 raft_consensus.cc:2229] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:56:58.824810 24733 ts_tablet_manager.cc:1905] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:56:58.825212 24734 raft_consensus.cc:2258] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:56:58.825791 24732 ts_tablet_manager.cc:1918] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:56:58.825938 24732 log.cc:1196] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/d1ecb1b9cd1740939d604189fa5fef00
I20260928 15:56:58.826409 24732 ts_tablet_manager.cc:1939] T d1ecb1b9cd1740939d604189fa5fef00 P a2e26707648940a2a35fd6c0bf5eba66: Deleting consensus metadata
I20260928 15:56:58.826958 24734 ts_tablet_manager.cc:1905] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:56:58.827620 24140 catalog_manager.cc:4841] TS a2e26707648940a2a35fd6c0bf5eba66 (127.22.125.193:37973): tablet d1ecb1b9cd1740939d604189fa5fef00 (table test-table [id=8406a5403c254f3bad3e135d959559e8]) successfully deleted
I20260928 15:56:58.831665 24733 ts_tablet_manager.cc:1918] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:56:58.831805 24733 log.cc:1196] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/d1ecb1b9cd1740939d604189fa5fef00
I20260928 15:56:58.832299 24733 ts_tablet_manager.cc:1939] T d1ecb1b9cd1740939d604189fa5fef00 P 0c69f561315d470181f7017a13b13a79: Deleting consensus metadata
I20260928 15:56:58.833355 24138 catalog_manager.cc:4841] TS 0c69f561315d470181f7017a13b13a79 (127.22.125.194:35805): tablet d1ecb1b9cd1740939d604189fa5fef00 (table test-table [id=8406a5403c254f3bad3e135d959559e8]) successfully deleted
I20260928 15:56:58.838212 24734 ts_tablet_manager.cc:1918] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:56:58.838385 24734 log.cc:1196] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/d1ecb1b9cd1740939d604189fa5fef00
I20260928 15:56:58.838868 24734 ts_tablet_manager.cc:1939] T d1ecb1b9cd1740939d604189fa5fef00 P ebff7dba3f264f59ac956c0e291fdc25: Deleting consensus metadata
I20260928 15:56:58.840251 24140 catalog_manager.cc:4841] TS ebff7dba3f264f59ac956c0e291fdc25 (127.22.125.195:35987): tablet d1ecb1b9cd1740939d604189fa5fef00 (table test-table [id=8406a5403c254f3bad3e135d959559e8]) successfully deleted
Sep 28 15:56:58 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:56:58.856200 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.822793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.22.125.193:37973)
0928 15:56:58.823035 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.823044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.829793 (+ 6749us) server_negotiation.cc:407] Connection header received
0928 15:56:58.830065 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.830070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.830159 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.830296 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:58.838547 (+ 8251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.839815 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.841936 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.842667 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.842817 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.848073 (+ 5256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:58.848093 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:58.848117 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:58.848160 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:58.852499 (+ 4339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.853452 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.853462 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.853466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.853624 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.854114 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.854119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.854123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.854647 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:58.854987 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.855316 (+ 329us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.855553 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260928 15:56:58.857095 24355 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38424
I20260928 15:56:58.880247 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.860422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.22.125.254:36909)
0928 15:56:58.860650 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.860656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.864542 (+ 3886us) server_negotiation.cc:407] Connection header received
0928 15:56:58.864715 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.864723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.864816 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.864950 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:58.866689 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.867857 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.869764 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.870468 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.870598 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.873628 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:58.873634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:58.873639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:58.873679 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:58.877225 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.878057 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.878062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.878066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.878204 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.878604 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.878609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.878612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.878976 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:58.879238 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.879632 (+ 394us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.879874 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
W20260928 15:56:58.880972 24154 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34140
I20260928 15:56:58.883740 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24225
I20260928 15:56:58.901760 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.886139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.22.125.193:37973)
0928 15:56:58.886350 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.886355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.886373 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:56:58.886451 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.886457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.886546 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.886647 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:58.888200 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.889345 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.891127 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.891856 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.891984 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.894729 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:58.894736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:58.894741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:58.894783 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:58.898701 (+ 3918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.899583 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.899588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.899591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.899768 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.900191 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.900196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.900200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.900517 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:58.900816 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.901111 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.901361 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260928 15:56:58.902767 24355 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38430
W20260928 15:56:58.903373 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24225: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:56:58.903545 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 24225
I20260928 15:56:58.926924 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24364
Sep 28 15:56:58 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:56:58.943244 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.927875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.22.125.194:35805)
0928 15:56:58.928833 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.928837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.928851 (+ 14us) server_negotiation.cc:407] Connection header received
0928 15:56:58.928903 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.928906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.928974 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.929099 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:58.930315 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.931067 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.932518 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.932938 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.933073 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.937167 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:58.937181 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:58.937195 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:58.937232 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:58.940421 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.941098 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.941105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.941107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.941264 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.941660 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.941664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.941667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.942012 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:58.942322 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.942627 (+ 305us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.942858 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":722,"threads_started":1}
W20260928 15:56:58.943979 24494 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58866
W20260928 15:56:58.944726 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24364: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:56:58.944912 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 24364
I20260928 15:56:58.962518 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24503
Sep 28 15:56:58 dist-test-slave-9cqd krb5kdc[24109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:56:58.978662 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:56:58.963386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.22.125.195:35987)
0928 15:56:58.963596 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 15:56:58.963602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:56:58.963621 (+ 19us) server_negotiation.cc:407] Connection header received
0928 15:56:58.963792 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:56:58.963798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:56:58.963878 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:56:58.963985 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:56:58.965207 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.966000 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.967231 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:56:58.967845 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:56:58.967988 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:56:58.972506 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:56:58.972520 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:56:58.972536 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:56:58.972580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:56:58.975619 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.976412 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.976420 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.976423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.976593 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:56:58.977000 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:56:58.977007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:56:58.977012 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:56:58.977459 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:56:58.977804 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0928 15:56:58.978089 (+ 285us) server_negotiation.cc:299] Negotiation successful
0928 15:56:58.978327 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260928 15:56:58.979344 24633 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49838
W20260928 15:56:58.980083 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24503: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:56:58.980230 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 24503
I20260928 15:56:58.999256 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24122
I20260928 15:56:59.000643 24194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:56:59.114944 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 24122
2026-09-28T15:56:59Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4871 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:56:59 dist-test-slave-9cqd krb5kdc[24744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:56:59 dist-test-slave-9cqd krb5kdc[24744](info): set up 2 sockets
Sep 28 15:56:59 dist-test-slave-9cqd krb5kdc[24744](info): commencing operation
krb5kdc: starting...
W20260928 15:57:00.194101 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:00 dist-test-slave-9cqd krb5kdc[24744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:00.375759 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:32825
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:32825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:00.694684 24757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:00.695016 24757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:00.695182 24757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:00.702391 24757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:00.702517 24757 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 15:57:00.702607 24757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:00.702710 24757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:00.702796 24757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:00.702878 24757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:00.712374 24757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:32825
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:32825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:00.713611 24757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:00.714875 24757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:00.726578 24762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:00.726532 24765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:00.726500 24763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:00.727586 24757 server_base.cc:1034] running on GCE node
I20260928 15:57:00.728323 24757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:00.729588 24757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:00.730834 24757 hybrid_clock.cc:648] HybridClock initialized: now 1790611020730781 us; error 47 us; skew 500 ppm
I20260928 15:57:00.731135 24757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:00 dist-test-slave-9cqd krb5kdc[24744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611020, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:00.734293 24757 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:00.735546 24757 webserver.cc:460] Webserver started at http://127.22.125.254:44709/ using document root <none> and password file <none>
I20260928 15:57:00.736224 24757 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:00.736310 24757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:00.736564 24757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:00.739243 24757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "8a1f53f01f134cf883f222b3884f4316"
format_stamp: "Formatted at 2026-09-28 15:57:00 on dist-test-slave-9cqd"
server_key: "088565df5ee647e45c980ce396f7702a"
server_key_iv: "2a13ebb9cc7356cd0adfaf97177e8860"
server_key_version: "encryptionkey@0"
I20260928 15:57:00.739923 24757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "8a1f53f01f134cf883f222b3884f4316"
format_stamp: "Formatted at 2026-09-28 15:57:00 on dist-test-slave-9cqd"
server_key: "088565df5ee647e45c980ce396f7702a"
server_key_iv: "2a13ebb9cc7356cd0adfaf97177e8860"
server_key_version: "encryptionkey@0"
I20260928 15:57:00.744742 24757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260928 15:57:00.747828 24772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:00.749074 24757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 15:57:00.749207 24757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "8a1f53f01f134cf883f222b3884f4316"
format_stamp: "Formatted at 2026-09-28 15:57:00 on dist-test-slave-9cqd"
server_key: "088565df5ee647e45c980ce396f7702a"
server_key_iv: "2a13ebb9cc7356cd0adfaf97177e8860"
server_key_version: "encryptionkey@0"
I20260928 15:57:00.749384 24757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:00.771003 24757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:00.775254 24757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:00.775475 24757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:00.796605 24757 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:32825
I20260928 15:57:00.796624 24833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:32825 every 8 connection(s)
I20260928 15:57:00.797971 24757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:00.801770 24834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:00.803082 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 24757
I20260928 15:57:00.803306 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:00.804180 23031 external_mini_cluster.cc:1428] Setting key 22af4ff574cc6dce76b226c9bcdd5a00
I20260928 15:57:00.816469 24834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Bootstrap starting.
Sep 28 15:57:00 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:00.820155 24834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:00.821332 24834 log.cc:824] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:00.824110 24834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: No bootstrap required, opened a new log
I20260928 15:57:00.826901 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:00.806258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.22.125.254:32825)
0928 15:57:00.807067 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:00.807077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:00.807121 (+ 44us) server_negotiation.cc:407] Connection header received
0928 15:57:00.808737 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:00.808759 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:00.809143 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:00.809475 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:00.811145 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:00.812463 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:00.814260 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:00.814893 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:00.815021 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:00.819685 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:00.819692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:00.819707 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:00.819742 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:00.822907 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:00.823597 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:00.823603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:00.823608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:00.823765 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:00.824157 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:00.824163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:00.824166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:00.824725 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:00.825010 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:00.825497 (+ 487us) server_negotiation.cc:299] Negotiation successful
0928 15:57:00.825910 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":410,"threads_started":1}
I20260928 15:57:00.832410 24834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1f53f01f134cf883f222b3884f4316" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 32825 } }
I20260928 15:57:00.832765 24834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:00.832871 24834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1f53f01f134cf883f222b3884f4316, State: Initialized, Role: FOLLOWER
I20260928 15:57:00.833417 24834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1f53f01f134cf883f222b3884f4316" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 32825 } }
I20260928 15:57:00.833660 24834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:00.833801 24834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:00.833986 24834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:00.836655 24834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1f53f01f134cf883f222b3884f4316" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 32825 } }
I20260928 15:57:00.837134 24834 leader_election.cc:302] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a1f53f01f134cf883f222b3884f4316; no voters:
I20260928 15:57:00.837678 24834 leader_election.cc:288] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:00.837805 24839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:00.838555 24839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [term 1 LEADER]: Becoming Leader. State: Replica: 8a1f53f01f134cf883f222b3884f4316, State: Running, Role: LEADER
I20260928 15:57:00.839056 24839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1f53f01f134cf883f222b3884f4316" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 32825 } }
I20260928 15:57:00.839702 24834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:00.841725 24841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a1f53f01f134cf883f222b3884f4316. Latest consensus state: current_term: 1 leader_uuid: "8a1f53f01f134cf883f222b3884f4316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1f53f01f134cf883f222b3884f4316" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 32825 } } }
I20260928 15:57:00.842113 24841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:00.842573 24840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a1f53f01f134cf883f222b3884f4316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1f53f01f134cf883f222b3884f4316" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 32825 } } }
I20260928 15:57:00.843003 24840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:00.850296 24845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:00.855163 24845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:00.864531 24845 catalog_manager.cc:1347] Generated new cluster ID: 63b8b44a413e499fa188b948e260c987
I20260928 15:57:00.864675 24845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:00.891950 24845 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:00.893528 24845 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:00.903611 24845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Generated new TSK 0
I20260928 15:57:00.904513 24845 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:57:00.951669 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:32825
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:01.298727 24860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:01.299034 24860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:01.299218 24860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:01.306572 24860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:01.306722 24860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:01.306861 24860 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 15:57:01.306951 24860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:57:01.307083 24860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:01.316776 24860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:01.318043 24860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:01.319384 24860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:01.333563 24865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:01.333509 24866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:01.333590 24868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:01.334043 24860 server_base.cc:1034] running on GCE node
I20260928 15:57:01.334805 24860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:01.335717 24860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:01.336910 24860 hybrid_clock.cc:648] HybridClock initialized: now 1790611021336854 us; error 54 us; skew 500 ppm
I20260928 15:57:01.337226 24860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:01 dist-test-slave-9cqd krb5kdc[24744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611021, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:01.340374 24860 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:57:01.341907 24860 webserver.cc:460] Webserver started at http://127.22.125.193:37013/ using document root <none> and password file <none>
I20260928 15:57:01.342525 24860 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:01.342624 24860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:01.342870 24860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:01.345819 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "c08d9273a1194d018dad3096c2ff81c0"
format_stamp: "Formatted at 2026-09-28 15:57:01 on dist-test-slave-9cqd"
server_key: "b4839cbaf5de126a408110fc71e92dd6"
server_key_iv: "81c707b2c7203ece26e34c54cfcadb2e"
server_key_version: "encryptionkey@0"
I20260928 15:57:01.346534 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "c08d9273a1194d018dad3096c2ff81c0"
format_stamp: "Formatted at 2026-09-28 15:57:01 on dist-test-slave-9cqd"
server_key: "b4839cbaf5de126a408110fc71e92dd6"
server_key_iv: "81c707b2c7203ece26e34c54cfcadb2e"
server_key_version: "encryptionkey@0"
I20260928 15:57:01.351661 24860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 15:57:01.355187 24875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:01.356541 24860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:01.356704 24860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "c08d9273a1194d018dad3096c2ff81c0"
format_stamp: "Formatted at 2026-09-28 15:57:01 on dist-test-slave-9cqd"
server_key: "b4839cbaf5de126a408110fc71e92dd6"
server_key_iv: "81c707b2c7203ece26e34c54cfcadb2e"
server_key_version: "encryptionkey@0"
I20260928 15:57:01.356910 24860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:01.372642 24860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:01.376430 24860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:01.376714 24860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:01.379311 24860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:01.379464 24860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:01.379587 24860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:01.379673 24860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:57:01 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611021, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:01.403497 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.379647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:35855 (local address 127.22.125.254:32825)
0928 15:57:01.380133 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:01.380140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:01.381456 (+ 1316us) server_negotiation.cc:407] Connection header received
0928 15:57:01.384478 (+ 3022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:01.384487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:01.384582 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:01.384704 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:01.387337 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.388308 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.390822 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.391370 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.391513 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.396674 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:01.396684 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:01.396689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:01.396733 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:01.400099 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.401081 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.401089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.401094 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.401239 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.401681 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.401688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.401692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.401971 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:01.402125 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:01.402896 (+ 771us) server_negotiation.cc:299] Negotiation successful
0928 15:57:01.403123 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":229,"threads_started":1}
I20260928 15:57:01.404848 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.380334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:32825 (local address 127.22.125.193:35855)
0928 15:57:01.381270 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:01.381332 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:01.384136 (+ 2804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:01.385020 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:01.385032 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:01.385958 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:01.387050 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.387083 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.388471 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.388480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.390673 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.390685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.391708 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.391715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.392156 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.392924 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:01.392966 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:01.396391 (+ 3425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:01.400249 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.400258 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.400274 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.400954 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.401354 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.401359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.401363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.401570 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.402100 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:01.402107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:01.402469 (+ 362us) client_negotiation.cc:769] Sending connection context
0928 15:57:01.402799 (+ 330us) client_negotiation.cc:240] Negotiation successful
0928 15:57:01.403319 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":347,"threads_started":1}
I20260928 15:57:01.425359 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.410659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:38527 (local address 127.22.125.254:32825)
0928 15:57:01.410813 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:01.410817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:01.411560 (+ 743us) server_negotiation.cc:407] Connection header received
0928 15:57:01.411770 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:01.411776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:01.411863 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:01.411968 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:01.413463 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.414151 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.416615 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.417062 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.417157 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.419513 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:01.419518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:01.419522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:01.419550 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:01.422700 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.423405 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.423411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.423414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.423537 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.423888 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.423895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.423900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.424223 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:01.424371 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:01.424787 (+ 416us) server_negotiation.cc:299] Negotiation successful
0928 15:57:01.425001 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 15:57:01.425465 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.410801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:32825 (local address 127.22.125.193:38527)
0928 15:57:01.411462 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:01.411478 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:01.411631 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:01.412028 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:01.412036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:01.412813 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:01.413304 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.413321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.414302 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.414310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.416447 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.416471 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.417271 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.417281 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.417548 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.418304 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:01.418321 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:01.419321 (+ 1000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:01.422822 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.422828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.422832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.423275 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.423646 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.423650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.423654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.423781 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.424340 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:01.424345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:01.424534 (+ 189us) client_negotiation.cc:769] Sending connection context
0928 15:57:01.424680 (+ 146us) client_negotiation.cc:240] Negotiation successful
0928 15:57:01.425008 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":351,"threads_started":1}
I20260928 15:57:01.439477 24860 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:36745
I20260928 15:57:01.439500 24995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:36745 every 8 connection(s)
I20260928 15:57:01.440930 24860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:01.442216 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 24860
I20260928 15:57:01.442457 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:01.443092 24996 heartbeater.cc:347] Connected to a master server at 127.22.125.254:32825
I20260928 15:57:01.443244 23031 external_mini_cluster.cc:1428] Setting key 9ea9b690dff438406aab3ad65bc307fc
I20260928 15:57:01.443347 24996 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:01.443976 24996 heartbeater.cc:511] Master 127.22.125.254:32825 requested a full tablet report, sending...
I20260928 15:57:01.446009 24789 ts_manager.cc:194] Registered new tserver with Master: c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745)
I20260928 15:57:01.447436 24789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:35855
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:57:01.483412 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:32825
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:01.802831 25000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:01.803141 25000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:01.803290 25000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:01.814590 25000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:01.814766 25000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:01.814935 25000 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 15:57:01.815038 25000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:57:01.815161 25000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:01.826323 25000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:01.827452 25000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:01.828751 25000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:01.841853 25005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:01.841809 25008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:01.841765 25006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:01.842644 25000 server_base.cc:1034] running on GCE node
I20260928 15:57:01.843225 25000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:01.844360 25000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:01.845597 25000 hybrid_clock.cc:648] HybridClock initialized: now 1790611021845539 us; error 45 us; skew 500 ppm
I20260928 15:57:01.845925 25000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:01 dist-test-slave-9cqd krb5kdc[24744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611021, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:01.849363 25000 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:57:01.850813 25000 webserver.cc:460] Webserver started at http://127.22.125.194:40171/ using document root <none> and password file <none>
I20260928 15:57:01.851403 25000 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:01.851500 25000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:01.851756 25000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:01.854738 25000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "e378d8e0acaf45419c16c07c462d8739"
format_stamp: "Formatted at 2026-09-28 15:57:01 on dist-test-slave-9cqd"
server_key: "775223c2fe6a4b1e4ac262e6f2d9ad0e"
server_key_iv: "adefcaf223dc1a584e51e86c6f46cc0e"
server_key_version: "encryptionkey@0"
I20260928 15:57:01.855417 25000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "e378d8e0acaf45419c16c07c462d8739"
format_stamp: "Formatted at 2026-09-28 15:57:01 on dist-test-slave-9cqd"
server_key: "775223c2fe6a4b1e4ac262e6f2d9ad0e"
server_key_iv: "adefcaf223dc1a584e51e86c6f46cc0e"
server_key_version: "encryptionkey@0"
I20260928 15:57:01.860606 25000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260928 15:57:01.863895 25015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:01.865154 25000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:01.865285 25000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "e378d8e0acaf45419c16c07c462d8739"
format_stamp: "Formatted at 2026-09-28 15:57:01 on dist-test-slave-9cqd"
server_key: "775223c2fe6a4b1e4ac262e6f2d9ad0e"
server_key_iv: "adefcaf223dc1a584e51e86c6f46cc0e"
server_key_version: "encryptionkey@0"
I20260928 15:57:01.865439 25000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:01.875909 25000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:01.879511 25000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:01.879739 25000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:01.882247 25000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:01.882378 25000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:01.882532 25000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:01.882627 25000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:57:01 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611021, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:01.908310 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.882679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:38511 (local address 127.22.125.254:32825)
0928 15:57:01.882916 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:01.882926 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:01.884268 (+ 1342us) server_negotiation.cc:407] Connection header received
0928 15:57:01.887156 (+ 2888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:01.887163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:01.887251 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:01.887368 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:01.889718 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.891022 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.893551 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.894203 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.894338 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.899720 (+ 5382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:01.899729 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:01.899737 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:01.899789 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:01.903811 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.905005 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.905012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.905015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.905182 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.905821 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.905830 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.905836 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.906303 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:01.906498 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:01.907812 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0928 15:57:01.907979 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 15:57:01.909904 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.883240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:32825 (local address 127.22.125.194:38511)
0928 15:57:01.884101 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:01.884148 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:01.886803 (+ 2655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:01.887790 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:01.887802 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:01.888661 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:01.889492 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.889514 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.891254 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.891262 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.893404 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.893415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.894442 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.894451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.894674 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.895452 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:01.895490 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:01.899410 (+ 3920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:01.904004 (+ 4594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.904012 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.904026 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.904792 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.905313 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.905324 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.905329 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.905650 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.906470 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:01.906481 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:01.906982 (+ 501us) client_negotiation.cc:769] Sending connection context
0928 15:57:01.907280 (+ 298us) client_negotiation.cc:240] Negotiation successful
0928 15:57:01.907993 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":338,"threads_started":1}
I20260928 15:57:01.929509 25000 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:42833
I20260928 15:57:01.929546 25134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:42833 every 8 connection(s)
I20260928 15:57:01.930763 25000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:01.931474 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 25000
I20260928 15:57:01.931649 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:01.932248 23031 external_mini_cluster.cc:1428] Setting key 5d7809e8d440613460e848ccd8f38724
I20260928 15:57:01.933275 25135 heartbeater.cc:347] Connected to a master server at 127.22.125.254:32825
I20260928 15:57:01.933365 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.917683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:32825 (local address 127.22.125.194:50247)
0928 15:57:01.918359 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:01.918377 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:01.918525 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:01.919045 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:01.919052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:01.919845 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:01.920273 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.920288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.921657 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.921667 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.923330 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:01.923341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.924683 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.924690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:01.924857 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.925724 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:01.925756 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:01.926973 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:01.930257 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.930263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.930267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.930632 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.931090 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:01.931094 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:01.931098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:01.931260 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:01.932230 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:01.932237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:01.932434 (+ 197us) client_negotiation.cc:769] Sending connection context
0928 15:57:01.932580 (+ 146us) client_negotiation.cc:240] Negotiation successful
0928 15:57:01.932893 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":512,"threads_started":1}
I20260928 15:57:01.933559 25135 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:01.934407 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:01.918151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:50247 (local address 127.22.125.254:32825)
0928 15:57:01.918387 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:01.918396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:01.918444 (+ 48us) server_negotiation.cc:407] Connection header received
0928 15:57:01.918649 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:01.918657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:01.918746 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:01.918865 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:01.920425 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.921416 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.923865 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:01.924483 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:01.924621 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:01.927155 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:01.927161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:01.927165 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:01.927209 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:01.930116 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.930775 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.930781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.930784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.930964 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:01.931409 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:01.931414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:01.931416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:01.931980 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:01.932167 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:01.933887 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0928 15:57:01.934109 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 15:57:01.934746 25135 heartbeater.cc:511] Master 127.22.125.254:32825 requested a full tablet report, sending...
I20260928 15:57:01.936450 24789 ts_manager.cc:194] Registered new tserver with Master: e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833)
I20260928 15:57:01.937222 24789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:38511
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:57:01.973377 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:32825
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:02.301100 25139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:02.301379 25139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:02.301486 25139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:02.308816 25139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:02.308915 25139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:02.308997 25139 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 15:57:02.309049 25139 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:57:02.309114 25139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:02.318815 25139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:02.319988 25139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:02.321401 25139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:02.334347 25144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:02.334441 25145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:02.334468 25147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:02.335178 25139 server_base.cc:1034] running on GCE node
I20260928 15:57:02.335744 25139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:02.336705 25139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:02.337900 25139 hybrid_clock.cc:648] HybridClock initialized: now 1790611022337864 us; error 31 us; skew 500 ppm
I20260928 15:57:02.338261 25139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:02 dist-test-slave-9cqd krb5kdc[24744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611022, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:02.341538 25139 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:57:02.342929 25139 webserver.cc:460] Webserver started at http://127.22.125.195:37457/ using document root <none> and password file <none>
I20260928 15:57:02.343593 25139 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:02.343693 25139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:02.343936 25139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:02.346729 25139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "a45cc58619d14ff0ae01146b8c09e264"
format_stamp: "Formatted at 2026-09-28 15:57:02 on dist-test-slave-9cqd"
server_key: "f54a9a9018c4553279f467f132d04329"
server_key_iv: "5c1b7c40191a54af7cb79a93fac2e7c2"
server_key_version: "encryptionkey@0"
I20260928 15:57:02.347416 25139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "a45cc58619d14ff0ae01146b8c09e264"
format_stamp: "Formatted at 2026-09-28 15:57:02 on dist-test-slave-9cqd"
server_key: "f54a9a9018c4553279f467f132d04329"
server_key_iv: "5c1b7c40191a54af7cb79a93fac2e7c2"
server_key_version: "encryptionkey@0"
I20260928 15:57:02.352626 25139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260928 15:57:02.356153 25154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:02.357507 25139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:02.357668 25139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "a45cc58619d14ff0ae01146b8c09e264"
format_stamp: "Formatted at 2026-09-28 15:57:02 on dist-test-slave-9cqd"
server_key: "f54a9a9018c4553279f467f132d04329"
server_key_iv: "5c1b7c40191a54af7cb79a93fac2e7c2"
server_key_version: "encryptionkey@0"
I20260928 15:57:02.357848 25139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:02.389523 25139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:02.393486 25139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:02.393731 25139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:02.396348 25139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:02.396483 25139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:02.396623 25139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:02.396713 25139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:57:02 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611022, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:02.421895 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.397168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:48417 (local address 127.22.125.254:32825)
0928 15:57:02.397345 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.397351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.398869 (+ 1518us) server_negotiation.cc:407] Connection header received
0928 15:57:02.402065 (+ 3196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.402074 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.402177 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.402295 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:02.404586 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.405918 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.408277 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.408948 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.409084 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.414329 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:02.414335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:02.414341 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:02.414382 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:02.417948 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:02.419030 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:02.419037 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:02.419041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:02.419203 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:02.419712 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:02.419718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:02.419721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:02.420038 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:02.420240 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.421286 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.421517 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 15:57:02.423249 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.397748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:32825 (local address 127.22.125.195:48417)
0928 15:57:02.398673 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:02.398723 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:02.401692 (+ 2969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:02.402597 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:02.402615 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:02.403472 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:02.404346 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.404365 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.406095 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.406104 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.408098 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.408117 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.409213 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.409223 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.409557 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.410400 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:02.410439 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:02.414112 (+ 3673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:02.418087 (+ 3975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:02.418096 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:02.418117 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:02.418827 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:02.419331 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:02.419336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:02.419339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:02.419582 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:02.420321 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:02.420329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:02.420800 (+ 471us) client_negotiation.cc:769] Sending connection context
0928 15:57:02.421171 (+ 371us) client_negotiation.cc:240] Negotiation successful
0928 15:57:02.421766 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":354,"threads_started":1}
I20260928 15:57:02.442492 25139 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:41547
I20260928 15:57:02.442519 25273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:41547 every 8 connection(s)
I20260928 15:57:02.443987 25139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:02.445814 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.430595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:32825 (local address 127.22.125.195:49037)
0928 15:57:02.431398 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:02.431418 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:02.431528 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:02.431937 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:02.431940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:02.432469 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:02.432961 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.432975 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.433961 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.433973 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.434988 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.435000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.435752 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.435758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.435926 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.436709 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:02.436726 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:02.437900 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:02.441965 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:02.441970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:02.441973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:02.442570 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:02.443422 (+ 852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:02.443427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:02.443430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:02.443590 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:02.444243 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:02.444249 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:02.444471 (+ 222us) client_negotiation.cc:769] Sending connection context
0928 15:57:02.444674 (+ 203us) client_negotiation.cc:240] Negotiation successful
0928 15:57:02.445132 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":480,"threads_started":1}
I20260928 15:57:02.446365 25274 heartbeater.cc:347] Connected to a master server at 127.22.125.254:32825
I20260928 15:57:02.446612 25274 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:02.447151 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.431226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:49037 (local address 127.22.125.254:32825)
0928 15:57:02.431460 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.431469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.431495 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:57:02.431654 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.431664 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.431781 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.431904 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:02.433114 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.433805 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.435129 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.435560 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.435696 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.438102 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:02.438106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:02.438110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:02.438139 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:02.441796 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:02.443122 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:02.443126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:02.443129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:02.443285 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:02.443732 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:02.443737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:02.443741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:02.444113 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:02.444267 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.446355 (+ 2088us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.446681 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 15:57:02.447352 25274 heartbeater.cc:511] Master 127.22.125.254:32825 requested a full tablet report, sending...
I20260928 15:57:02.449012 24789 ts_manager.cc:194] Registered new tserver with Master: a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:02.450182 24996 heartbeater.cc:503] Master 127.22.125.254:32825 was elected leader, sending a full tablet report...
I20260928 15:57:02.450958 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 25139
I20260928 15:57:02.451123 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:02.451601 23031 external_mini_cluster.cc:1428] Setting key df60b0ba32ee7f1853de4ddb18fa6903
I20260928 15:57:02.451602 24789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:48417
I20260928 15:57:02.454877 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 15:57:02 dist-test-slave-9cqd krb5kdc[24744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 15:57:02 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:02.502117 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.487481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.22.125.254:32825)
0928 15:57:02.487720 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.487727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.487901 (+ 174us) server_negotiation.cc:407] Connection header received
0928 15:57:02.488158 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.488162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.488228 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.488336 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:02.489705 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.490711 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.492593 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.493097 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.493221 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.497045 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:02.497050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:02.497053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:02.497086 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:02.499706 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:02.500382 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:02.500387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:02.500389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:02.500509 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:02.500835 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:02.500838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:02.500840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:02.501105 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:02.501299 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.501555 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.501718 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 15:57:02.506557 24788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:02.513557 24788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:02.539248 25288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.526886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.22.125.195:41547)
0928 15:57:02.527648 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.527657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.529679 (+ 2022us) server_negotiation.cc:407] Connection header received
0928 15:57:02.529784 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.529792 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.530066 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.530291 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:02.531487 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.533376 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.536370 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.537668 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.537853 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.538044 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.538712 (+ 668us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.538890 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":388,"threads_started":1}
I20260928 15:57:02.539305 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.526559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:41547 (local address 127.0.0.1:56282)
0928 15:57:02.529292 (+ 2733us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:02.529329 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:02.529504 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:02.530632 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:02.530643 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:02.530684 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:02.531197 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.531214 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.533535 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.533541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.536096 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.536120 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.538204 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.538211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.538444 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.538504 (+ 60us) client_negotiation.cc:769] Sending connection context
0928 15:57:02.538642 (+ 138us) client_negotiation.cc:240] Negotiation successful
0928 15:57:02.538827 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2496,"mutex_wait_us":3,"thread_start_us":2528,"threads_started":1}
I20260928 15:57:02.542488 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.533926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.22.125.193:36745)
0928 15:57:02.534501 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.534511 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.535199 (+ 688us) server_negotiation.cc:407] Connection header received
0928 15:57:02.535353 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.535360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.535587 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.535772 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:02.536515 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.537882 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.540315 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.541485 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.541674 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.541835 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.542048 (+ 213us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.542198 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":263,"threads_started":1}
I20260928 15:57:02.542558 25292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.534515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:36745 (local address 127.0.0.1:40838)
0928 15:57:02.535067 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:02.535088 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:02.535210 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:02.535760 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:02.535765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:02.535789 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:02.536281 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.536294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.538089 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.538095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.540163 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.540176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.541683 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.541689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.541857 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.541890 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 15:57:02.542006 (+ 116us) client_negotiation.cc:240] Negotiation successful
0928 15:57:02.542131 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":25,"thread_start_us":287,"threads_started":1}
I20260928 15:57:02.543646 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.528837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:42833 (local address 127.0.0.1:41282)
0928 15:57:02.533579 (+ 4742us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:02.533603 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:02.533720 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:02.534858 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:02.534866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:02.534890 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:02.535352 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.535365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.537160 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.537165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.540540 (+ 3375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.540560 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.542510 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.542518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.543027 (+ 509us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.543086 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 15:57:02.543176 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 15:57:02.543298 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4580,"mutex_wait_us":33,"thread_start_us":4393,"threads_started":1}
I20260928 15:57:02.544435 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.529874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.22.125.194:42833)
0928 15:57:02.533965 (+ 4091us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.533975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.534001 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:57:02.534104 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.534113 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.534355 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.534601 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:02.535536 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.537012 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.540689 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.542033 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.542199 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.542347 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.543960 (+ 1613us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.544149 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2952,"thread_start_us":2796,"threads_started":1}
I20260928 15:57:02.543704 25204 tablet_service.cc:1467] Processing CreateTablet for tablet f5749d4a8f5e440d9975e1e8f0a4878f (DEFAULT_TABLE table=test-table [id=9a2887f55f6e42a0bf3b1a7ffba082f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:02.545918 25204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5749d4a8f5e440d9975e1e8f0a4878f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:02.546032 24926 tablet_service.cc:1467] Processing CreateTablet for tablet f5749d4a8f5e440d9975e1e8f0a4878f (DEFAULT_TABLE table=test-table [id=9a2887f55f6e42a0bf3b1a7ffba082f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:02.547411 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5749d4a8f5e440d9975e1e8f0a4878f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:02.547782 25065 tablet_service.cc:1467] Processing CreateTablet for tablet f5749d4a8f5e440d9975e1e8f0a4878f (DEFAULT_TABLE table=test-table [id=9a2887f55f6e42a0bf3b1a7ffba082f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:02.549074 25065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5749d4a8f5e440d9975e1e8f0a4878f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:02.558640 25293 tablet_bootstrap.cc:492] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Bootstrap starting.
I20260928 15:57:02.561909 25293 tablet_bootstrap.cc:654] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:02.562208 25294 tablet_bootstrap.cc:492] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Bootstrap starting.
I20260928 15:57:02.563225 25293 log.cc:824] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:02.563413 25295 tablet_bootstrap.cc:492] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Bootstrap starting.
I20260928 15:57:02.566030 25294 tablet_bootstrap.cc:654] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:02.566479 25295 tablet_bootstrap.cc:654] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:02.567231 25293 tablet_bootstrap.cc:492] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: No bootstrap required, opened a new log
I20260928 15:57:02.567373 25294 log.cc:824] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:02.567505 25293 ts_tablet_manager.cc:1397] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260928 15:57:02.567662 25295 log.cc:824] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:02.570302 25294 tablet_bootstrap.cc:492] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: No bootstrap required, opened a new log
I20260928 15:57:02.570565 25294 ts_tablet_manager.cc:1397] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260928 15:57:02.570591 25295 tablet_bootstrap.cc:492] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: No bootstrap required, opened a new log
I20260928 15:57:02.570824 25295 ts_tablet_manager.cc:1397] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260928 15:57:02.575021 25293 raft_consensus.cc:348] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.575366 25293 raft_consensus.cc:374] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:02.575479 25293 raft_consensus.cc:729] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45cc58619d14ff0ae01146b8c09e264, State: Initialized, Role: FOLLOWER
I20260928 15:57:02.576149 25293 consensus_queue.cc:260] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.578554 25274 heartbeater.cc:503] Master 127.22.125.254:32825 was elected leader, sending a full tablet report...
I20260928 15:57:02.580430 25293 ts_tablet_manager.cc:1428] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260928 15:57:02.581133 25295 raft_consensus.cc:348] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.581127 25294 raft_consensus.cc:348] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.581525 25294 raft_consensus.cc:374] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:02.581525 25295 raft_consensus.cc:374] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:02.581627 25294 raft_consensus.cc:729] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Initialized, Role: FOLLOWER
I20260928 15:57:02.581626 25295 raft_consensus.cc:729] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e378d8e0acaf45419c16c07c462d8739, State: Initialized, Role: FOLLOWER
I20260928 15:57:02.582202 25295 consensus_queue.cc:260] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.582202 25294 consensus_queue.cc:260] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.583843 25295 ts_tablet_manager.cc:1428] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260928 15:57:02.584344 25294 ts_tablet_manager.cc:1428] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260928 15:57:02.585004 25135 heartbeater.cc:503] Master 127.22.125.254:32825 was elected leader, sending a full tablet report...
W20260928 15:57:02.687264 25136 tablet.cc:2369] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:57:02.693547 24997 tablet.cc:2369] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:57:02.702304 25275 tablet.cc:2369] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:57:02.941489 25300 raft_consensus.cc:482] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:02.941749 25300 raft_consensus.cc:504] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.943369 25300 leader_election.cc:288] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547), e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833)
I20260928 15:57:02.950665 25288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.943726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:51139 (local address 127.22.125.195:41547)
0928 15:57:02.943921 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.943929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.945096 (+ 1167us) server_negotiation.cc:407] Connection header received
0928 15:57:02.945307 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.945313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.945385 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.945487 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:02.946094 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.946852 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.948863 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.949700 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.949826 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.949877 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.950256 (+ 379us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.950402 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 15:57:02.950731 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.944133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:55929 (local address 127.22.125.194:42833)
0928 15:57:02.944365 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:02.944389 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:02.945138 (+ 749us) server_negotiation.cc:407] Connection header received
0928 15:57:02.945328 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:02.945332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:02.945387 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:02.945483 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:02.946214 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.947030 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.949056 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.949849 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.950027 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.950095 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:02.950362 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 15:57:02.950503 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 15:57:02.950742 25303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.943811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:41547 (local address 127.22.125.193:51139)
0928 15:57:02.944984 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:02.945000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:02.945148 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:02.945501 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:02.945506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:02.945538 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:02.945906 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.945918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.946994 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.947000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.948715 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.948726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.949904 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.949912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.950082 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.950118 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 15:57:02.950201 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 15:57:02.950309 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":25,"thread_start_us":893,"threads_started":1}
I20260928 15:57:02.951233 25302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:02.943724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:42833 (local address 127.22.125.193:55929)
0928 15:57:02.945005 (+ 1281us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:02.945020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:02.945178 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:02.945502 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:02.945506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:02.945593 (+ 87us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:02.946024 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.946038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.947171 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.947174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.948869 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:02.948882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:02.950062 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:02.950066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:02.950194 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:02.950231 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 15:57:02.950321 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 15:57:02.950451 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"spinlock_wait_cycles":20736,"thread_start_us":772,"threads_started":1}
I20260928 15:57:02.951749 25224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5749d4a8f5e440d9975e1e8f0a4878f" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264" is_pre_election: true
I20260928 15:57:02.952330 25224 raft_consensus.cc:2454] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 0.
I20260928 15:57:02.952268 25085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5749d4a8f5e440d9975e1e8f0a4878f" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e378d8e0acaf45419c16c07c462d8739" is_pre_election: true
I20260928 15:57:02.952764 25085 raft_consensus.cc:2454] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 0.
I20260928 15:57:02.952968 24878 leader_election.cc:302] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45cc58619d14ff0ae01146b8c09e264, c08d9273a1194d018dad3096c2ff81c0; no voters:
I20260928 15:57:02.953378 25300 raft_consensus.cc:2781] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:02.953531 25300 raft_consensus.cc:482] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:02.953625 25300 raft_consensus.cc:3037] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:02.955735 25300 raft_consensus.cc:504] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.956408 25300 leader_election.cc:288] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 election: Requested vote from peers a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547), e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833)
I20260928 15:57:02.956805 25224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5749d4a8f5e440d9975e1e8f0a4878f" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264"
I20260928 15:57:02.956871 25085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5749d4a8f5e440d9975e1e8f0a4878f" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e378d8e0acaf45419c16c07c462d8739"
I20260928 15:57:02.957000 25224 raft_consensus.cc:3037] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:02.957089 25085 raft_consensus.cc:3037] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:02.959022 25224 raft_consensus.cc:2454] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 1.
I20260928 15:57:02.959429 25085 raft_consensus.cc:2454] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 1.
I20260928 15:57:02.959424 24878 leader_election.cc:302] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45cc58619d14ff0ae01146b8c09e264, c08d9273a1194d018dad3096c2ff81c0; no voters:
I20260928 15:57:02.959731 25300 raft_consensus.cc:2781] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:02.960222 25300 raft_consensus.cc:686] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 1 LEADER]: Becoming Leader. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Running, Role: LEADER
I20260928 15:57:02.960785 25300 consensus_queue.cc:237] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } }
I20260928 15:57:02.965230 24789 catalog_manager.cc:5495] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193). New cstate: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } health_report { overall_health: HEALTHY } } }
Sep 28 15:57:03 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611020, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:03.023178 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.010256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:32825 (local address 127.0.0.1:40902)
0928 15:57:03.011004 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:03.011024 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:03.011148 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:03.011449 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:03.011452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:03.012228 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:03.012517 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.012528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.013648 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.013652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.014575 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.014586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.015251 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.015255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.015378 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.016087 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:03.016108 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:03.018199 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:03.021085 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:03.021096 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:03.021101 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:03.021514 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:03.021813 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:03.021816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:03.021818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:03.021940 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:03.022376 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:03.022382 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:03.022524 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 15:57:03.022651 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 15:57:03.022837 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":398,"threads_started":1}
I20260928 15:57:03.023344 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.010280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.22.125.254:32825)
0928 15:57:03.010914 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.010919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.011101 (+ 182us) server_negotiation.cc:407] Connection header received
0928 15:57:03.011247 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.011251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.011308 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.011410 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:03.012671 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.013538 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.014690 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.015097 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.015186 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.018387 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:03.018392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:03.018397 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:03.018425 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:03.020965 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:03.021611 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:03.021615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:03.021617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:03.021725 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:03.022026 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:03.022029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:03.022031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:03.022273 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:03.022381 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.022737 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.022980 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":504,"thread_start_us":336,"threads_started":1}
I20260928 15:57:03.028475 24789 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:40902:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 15:57:03.029486 24789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:03.036104 25065 tablet_service.cc:1467] Processing CreateTablet for tablet a7926efd6d074e81999d2d80ce703f58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2e8a12ab0d048988056217d202fcc2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:57:03.036024 24926 tablet_service.cc:1467] Processing CreateTablet for tablet a7926efd6d074e81999d2d80ce703f58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2e8a12ab0d048988056217d202fcc2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:57:03.036396 25204 tablet_service.cc:1467] Processing CreateTablet for tablet a7926efd6d074e81999d2d80ce703f58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2e8a12ab0d048988056217d202fcc2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:57:03.037065 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7926efd6d074e81999d2d80ce703f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:03.037065 25065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7926efd6d074e81999d2d80ce703f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:03.037202 25204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7926efd6d074e81999d2d80ce703f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:03.043254 25293 tablet_bootstrap.cc:492] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: Bootstrap starting.
I20260928 15:57:03.044250 25294 tablet_bootstrap.cc:492] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: Bootstrap starting.
I20260928 15:57:03.044268 25295 tablet_bootstrap.cc:492] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: Bootstrap starting.
I20260928 15:57:03.044866 25293 tablet_bootstrap.cc:654] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:03.045477 25294 tablet_bootstrap.cc:654] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:03.045995 25295 tablet_bootstrap.cc:654] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:03.047983 25293 tablet_bootstrap.cc:492] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: No bootstrap required, opened a new log
I20260928 15:57:03.048237 25293 ts_tablet_manager.cc:1397] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 15:57:03.048276 25294 tablet_bootstrap.cc:492] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: No bootstrap required, opened a new log
I20260928 15:57:03.048424 25294 ts_tablet_manager.cc:1397] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260928 15:57:03.049157 25295 tablet_bootstrap.cc:492] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: No bootstrap required, opened a new log
I20260928 15:57:03.049013 25293 raft_consensus.cc:348] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.049273 25293 raft_consensus.cc:374] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:03.049306 25295 ts_tablet_manager.cc:1397] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260928 15:57:03.049372 25293 raft_consensus.cc:729] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45cc58619d14ff0ae01146b8c09e264, State: Initialized, Role: FOLLOWER
I20260928 15:57:03.049640 25293 consensus_queue.cc:260] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.049189 25294 raft_consensus.cc:348] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.049978 25294 raft_consensus.cc:374] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:03.050084 25294 raft_consensus.cc:729] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Initialized, Role: FOLLOWER
I20260928 15:57:03.050024 25295 raft_consensus.cc:348] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.050253 25295 raft_consensus.cc:374] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:03.050062 25299 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.050343 25295 raft_consensus.cc:729] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e378d8e0acaf45419c16c07c462d8739, State: Initialized, Role: FOLLOWER
I20260928 15:57:03.050367 25299 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:03.050334 25294 consensus_queue.cc:260] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.050561 25295 consensus_queue.cc:260] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.050848 25300 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.051103 25300 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:03.051156 25295 ts_tablet_manager.cc:1428] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 15:57:03.051216 25301 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.051597 25301 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:03.051631 25294 ts_tablet_manager.cc:1428] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 15:57:03.052810 25293 ts_tablet_manager.cc:1428] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260928 15:57:03.194017 25300 raft_consensus.cc:482] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:03.194271 25300 raft_consensus.cc:504] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.194885 25300 leader_election.cc:288] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:03.195492 25224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7926efd6d074e81999d2d80ce703f58" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264" is_pre_election: true
I20260928 15:57:03.195492 25085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7926efd6d074e81999d2d80ce703f58" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e378d8e0acaf45419c16c07c462d8739" is_pre_election: true
I20260928 15:57:03.195767 25224 raft_consensus.cc:2454] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 0.
I20260928 15:57:03.195766 25085 raft_consensus.cc:2454] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 0.
I20260928 15:57:03.196234 24879 leader_election.cc:302] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a45cc58619d14ff0ae01146b8c09e264, c08d9273a1194d018dad3096c2ff81c0, e378d8e0acaf45419c16c07c462d8739; no voters:
I20260928 15:57:03.196581 25300 raft_consensus.cc:2781] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:03.196695 25300 raft_consensus.cc:482] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:03.196825 25300 raft_consensus.cc:3037] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:03.198814 25300 raft_consensus.cc:504] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.199357 25300 leader_election.cc:288] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 election: Requested vote from peers e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:03.199805 25085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7926efd6d074e81999d2d80ce703f58" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e378d8e0acaf45419c16c07c462d8739"
I20260928 15:57:03.199805 25224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7926efd6d074e81999d2d80ce703f58" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264"
I20260928 15:57:03.200028 25085 raft_consensus.cc:3037] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:03.200029 25224 raft_consensus.cc:3037] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:03.202929 25085 raft_consensus.cc:2454] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 1.
I20260928 15:57:03.202931 25224 raft_consensus.cc:2454] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 1.
I20260928 15:57:03.203416 24879 leader_election.cc:302] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c08d9273a1194d018dad3096c2ff81c0, e378d8e0acaf45419c16c07c462d8739; no voters:
I20260928 15:57:03.203675 25300 raft_consensus.cc:2781] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:03.203862 25300 raft_consensus.cc:686] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [term 1 LEADER]: Becoming Leader. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Running, Role: LEADER
I20260928 15:57:03.204141 25300 consensus_queue.cc:237] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.205535 25304 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: TxnStatusTablet state changed. Reason: New leader c08d9273a1194d018dad3096c2ff81c0. Latest consensus state: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.205801 25304 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:57:03.207357 24788 catalog_manager.cc:5495] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193). New cstate: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:03.208496 25317 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 15:57:03.208781 25317 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 15:57:03.241469 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.234758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.22.125.193:36745)
0928 15:57:03.235364 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.235369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.235393 (+ 24us) server_negotiation.cc:407] Connection header received
0928 15:57:03.235468 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.235472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.235546 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.235673 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:03.236160 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.237025 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.238948 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.239373 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.239470 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.240118 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.240682 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.240907 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.241057 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.241124 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":342,"threads_started":1}
I20260928 15:57:03.241521 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.234675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:36745 (local address 127.0.0.1:40842)
0928 15:57:03.234923 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:03.234944 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:03.235086 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:03.235729 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:03.235735 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:03.235749 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:03.235945 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.235957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.237157 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.237163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.238794 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.238810 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.239526 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.239532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.239772 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.239844 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.240891 (+ 1047us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.240914 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 15:57:03.241016 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 15:57:03.241188 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260928 15:57:03.249045 25085 raft_consensus.cc:1261] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Refusing update from remote peer c08d9273a1194d018dad3096c2ff81c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:03.249230 25224 raft_consensus.cc:1261] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Refusing update from remote peer c08d9273a1194d018dad3096c2ff81c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:03.250038 25304 consensus_queue.cc:1035] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:03.250407 25300 consensus_queue.cc:1035] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:03.261408 25301 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: TxnStatusTablet state changed. Reason: New leader c08d9273a1194d018dad3096c2ff81c0. Latest consensus state: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.261726 25301 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:03.264806 25304 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.265118 25304 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:57:03.265072 25322 mvcc.cc:205] Tried to move back new op lower bound from 7334342751217922048 to 7334342751046148096. Current Snapshot: MvccSnapshot[applied={T|T < 7334342751217922048}]
I20260928 15:57:03.265514 25321 mvcc.cc:205] Tried to move back new op lower bound from 7334342751217922048 to 7334342751046148096. Current Snapshot: MvccSnapshot[applied={T|T < 7334342751217922048}]
I20260928 15:57:03.269227 25301 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.269434 25301 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P e378d8e0acaf45419c16c07c462d8739: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:03.274628 25299 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: TxnStatusTablet state changed. Reason: New leader c08d9273a1194d018dad3096c2ff81c0. Latest consensus state: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.274924 25299 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:03.277401 25300 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.277747 25300 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P c08d9273a1194d018dad3096c2ff81c0: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:57:03.284354 25323 mvcc.cc:205] Tried to move back new op lower bound from 7334342751217922048 to 7334342751046148096. Current Snapshot: MvccSnapshot[applied={T|T < 7334342751217922048}]
I20260928 15:57:03.286064 25299 tablet_replica.cc:439] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } } }
I20260928 15:57:03.286347 25299 tablet_replica.cc:442] T a7926efd6d074e81999d2d80ce703f58 P a45cc58619d14ff0ae01146b8c09e264: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:03.298211 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.292744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.22.125.193:36745)
0928 15:57:03.292928 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.292935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.293383 (+ 448us) server_negotiation.cc:407] Connection header received
0928 15:57:03.293604 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.293610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.293686 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.293787 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:03.294262 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.295006 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.296431 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.296895 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.297028 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.297469 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.297572 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.297673 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.297824 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.297903 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 15:57:03.316087 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.309422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:36745 (local address 127.22.125.193:38033)
0928 15:57:03.310116 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:03.310136 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:03.310244 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:03.310587 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:03.310593 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:03.310602 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:03.310951 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.310959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.312461 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.312467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.313926 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.313938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.314916 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.314920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.315033 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.315081 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.315506 (+ 425us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.315514 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 15:57:03.315578 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 15:57:03.315674 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":379,"threads_started":1}
I20260928 15:57:03.316092 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.309356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:38033 (local address 127.22.125.193:36745)
0928 15:57:03.309555 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.309561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.310215 (+ 654us) server_negotiation.cc:407] Connection header received
0928 15:57:03.310354 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.310359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.310469 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.310569 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:03.311098 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.312310 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.314057 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.314752 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.314872 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.315283 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.315402 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.315499 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.315634 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.315718 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 15:57:03.325997 25085 raft_consensus.cc:1261] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Refusing update from remote peer c08d9273a1194d018dad3096c2ff81c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:03.326200 25223 raft_consensus.cc:1261] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Refusing update from remote peer c08d9273a1194d018dad3096c2ff81c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:03.326843 25329 consensus_queue.cc:1035] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:03.327227 25304 consensus_queue.cc:1035] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:03.387102 24788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:03.387380 24788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:03.390928 24788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Sending DeleteTablet for 3 replicas of tablet f5749d4a8f5e440d9975e1e8f0a4878f
I20260928 15:57:03.392208 25204 tablet_service.cc:1514] Processing DeleteTablet for tablet f5749d4a8f5e440d9975e1e8f0a4878f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:03 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:56282
I20260928 15:57:03.392545 25065 tablet_service.cc:1514] Processing DeleteTablet for tablet f5749d4a8f5e440d9975e1e8f0a4878f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:03 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:41282
I20260928 15:57:03.392963 24925 tablet_service.cc:1514] Processing DeleteTablet for tablet f5749d4a8f5e440d9975e1e8f0a4878f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:03 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:40838
I20260928 15:57:03.393503 25339 tablet_replica.cc:330] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: stopping tablet replica
I20260928 15:57:03.394021 25339 raft_consensus.cc:2229] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:03.394471 25339 raft_consensus.cc:2258] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:03.394325 24788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:03.394994 24788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:03.395615 25340 tablet_replica.cc:330] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: stopping tablet replica
I20260928 15:57:03.396135 25340 raft_consensus.cc:2229] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:03.397082 25338 tablet_replica.cc:330] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: stopping tablet replica
I20260928 15:57:03.397493 25338 raft_consensus.cc:2229] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:03.397866 25338 raft_consensus.cc:2258] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:03.397914 25339 ts_tablet_manager.cc:1905] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:03.399787 25338 ts_tablet_manager.cc:1905] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:03.404176 25065 tablet_service.cc:1467] Processing CreateTablet for tablet d5497c45900f41fe8ae15f40785d13c2 (DEFAULT_TABLE table=test-table [id=296fe47e98664af29b01aeb26a5f41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:03.404284 25203 tablet_service.cc:1467] Processing CreateTablet for tablet d5497c45900f41fe8ae15f40785d13c2 (DEFAULT_TABLE table=test-table [id=296fe47e98664af29b01aeb26a5f41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:03.404616 25065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5497c45900f41fe8ae15f40785d13c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:03.404810 25203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5497c45900f41fe8ae15f40785d13c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:03.405238 25340 raft_consensus.cc:2258] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:03.407290 25340 ts_tablet_manager.cc:1905] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:03.408885 25339 ts_tablet_manager.cc:1918] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:03.408205 24925 tablet_service.cc:1467] Processing CreateTablet for tablet d5497c45900f41fe8ae15f40785d13c2 (DEFAULT_TABLE table=test-table [id=296fe47e98664af29b01aeb26a5f41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:03.411698 25295 tablet_bootstrap.cc:492] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: Bootstrap starting.
I20260928 15:57:03.412201 25339 log.cc:1196] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/f5749d4a8f5e440d9975e1e8f0a4878f
I20260928 15:57:03.412379 24925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5497c45900f41fe8ae15f40785d13c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:03.412967 25339 ts_tablet_manager.cc:1939] T f5749d4a8f5e440d9975e1e8f0a4878f P e378d8e0acaf45419c16c07c462d8739: Deleting consensus metadata
I20260928 15:57:03.413911 25295 tablet_bootstrap.cc:654] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:03.415052 24776 catalog_manager.cc:4841] TS e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833): tablet f5749d4a8f5e440d9975e1e8f0a4878f (table test-table [id=9a2887f55f6e42a0bf3b1a7ffba082f3]) successfully deleted
I20260928 15:57:03.417299 25293 tablet_bootstrap.cc:492] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: Bootstrap starting.
I20260928 15:57:03.418959 25293 tablet_bootstrap.cc:654] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:03.420104 25294 tablet_bootstrap.cc:492] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: Bootstrap starting.
I20260928 15:57:03.420617 25340 ts_tablet_manager.cc:1918] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:03.420863 25340 log.cc:1196] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/f5749d4a8f5e440d9975e1e8f0a4878f
I20260928 15:57:03.421554 25340 ts_tablet_manager.cc:1939] T f5749d4a8f5e440d9975e1e8f0a4878f P c08d9273a1194d018dad3096c2ff81c0: Deleting consensus metadata
I20260928 15:57:03.421793 25294 tablet_bootstrap.cc:654] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:03.423600 24776 catalog_manager.cc:4841] TS c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745): tablet f5749d4a8f5e440d9975e1e8f0a4878f (table test-table [id=9a2887f55f6e42a0bf3b1a7ffba082f3]) successfully deleted
I20260928 15:57:03.437582 25338 ts_tablet_manager.cc:1918] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:03.437793 25338 log.cc:1196] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/f5749d4a8f5e440d9975e1e8f0a4878f
I20260928 15:57:03.438376 25338 ts_tablet_manager.cc:1939] T f5749d4a8f5e440d9975e1e8f0a4878f P a45cc58619d14ff0ae01146b8c09e264: Deleting consensus metadata
I20260928 15:57:03.440204 24775 catalog_manager.cc:4841] TS a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547): tablet f5749d4a8f5e440d9975e1e8f0a4878f (table test-table [id=9a2887f55f6e42a0bf3b1a7ffba082f3]) successfully deleted
I20260928 15:57:03.446359 25293 tablet_bootstrap.cc:492] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: No bootstrap required, opened a new log
I20260928 15:57:03.446628 25293 ts_tablet_manager.cc:1397] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.005s
I20260928 15:57:03.447439 25293 raft_consensus.cc:348] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.447806 25293 raft_consensus.cc:374] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:03.447922 25293 raft_consensus.cc:729] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45cc58619d14ff0ae01146b8c09e264, State: Initialized, Role: FOLLOWER
I20260928 15:57:03.448249 25293 consensus_queue.cc:260] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.448959 25293 ts_tablet_manager.cc:1428] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:03.450502 25295 tablet_bootstrap.cc:492] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: No bootstrap required, opened a new log
I20260928 15:57:03.450717 25295 ts_tablet_manager.cc:1397] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.006s
I20260928 15:57:03.451707 25295 raft_consensus.cc:348] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.452000 25295 raft_consensus.cc:374] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:03.452167 25295 raft_consensus.cc:729] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e378d8e0acaf45419c16c07c462d8739, State: Initialized, Role: FOLLOWER
I20260928 15:57:03.452564 25295 consensus_queue.cc:260] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.453171 25295 ts_tablet_manager.cc:1428] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:03.454843 25294 tablet_bootstrap.cc:492] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: No bootstrap required, opened a new log
I20260928 15:57:03.455037 25294 ts_tablet_manager.cc:1397] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: Time spent bootstrapping tablet: real 0.035s user 0.015s sys 0.014s
I20260928 15:57:03.455772 25294 raft_consensus.cc:348] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.456099 25294 raft_consensus.cc:374] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:03.456246 25294 raft_consensus.cc:729] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Initialized, Role: FOLLOWER
I20260928 15:57:03.456528 25294 consensus_queue.cc:260] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.457243 25294 ts_tablet_manager.cc:1428] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260928 15:57:03.567692 25301 raft_consensus.cc:482] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:03.567960 25301 raft_consensus.cc:504] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.569519 25301 leader_election.cc:288] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:03.588438 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.570201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:33945 (local address 127.22.125.193:36745)
0928 15:57:03.570421 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.570427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.579936 (+ 9509us) server_negotiation.cc:407] Connection header received
0928 15:57:03.580262 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.580271 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.580404 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.580494 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:03.581445 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.582869 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.585935 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.587350 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.587532 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.587611 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.587969 (+ 358us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.588158 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 15:57:03.589353 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.569758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:36745 (local address 127.22.125.194:33945)
0928 15:57:03.579774 (+ 10016us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:03.579799 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:03.580003 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:03.580793 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:03.580800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:03.580840 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:03.581186 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.581201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.583075 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.583081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.585681 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.585696 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.587538 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.587548 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.587755 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.587792 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 15:57:03.588762 (+ 970us) client_negotiation.cc:240] Negotiation successful
0928 15:57:03.588916 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9832,"thread_start_us":9758,"threads_started":1}
I20260928 15:57:03.590387 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5497c45900f41fe8ae15f40785d13c2" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c08d9273a1194d018dad3096c2ff81c0" is_pre_election: true
I20260928 15:57:03.590687 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.569933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:33305 (local address 127.22.125.195:41547)
0928 15:57:03.581767 (+ 11834us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.581774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.583603 (+ 1829us) server_negotiation.cc:407] Connection header received
0928 15:57:03.583828 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.583832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.583896 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.584006 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:03.584820 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.585667 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.587946 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.588930 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.589094 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.589173 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.590209 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.590349 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11702,"thread_start_us":11591,"threads_started":1}
I20260928 15:57:03.590950 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.569836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:41547 (local address 127.22.125.194:33305)
0928 15:57:03.583498 (+ 13662us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:03.583514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:03.583649 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:03.584094 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:03.584099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:03.584123 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:03.584631 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.584641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.585786 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.585793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.587791 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.587803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.589157 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.589163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.589356 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.589385 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 15:57:03.590184 (+ 799us) client_negotiation.cc:240] Negotiation successful
0928 15:57:03.590315 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13537,"mutex_wait_us":28,"thread_start_us":13316,"threads_started":1}
I20260928 15:57:03.590895 24951 raft_consensus.cc:2454] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 0.
I20260928 15:57:03.591573 25019 leader_election.cc:302] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c08d9273a1194d018dad3096c2ff81c0, e378d8e0acaf45419c16c07c462d8739; no voters:
I20260928 15:57:03.591673 25223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5497c45900f41fe8ae15f40785d13c2" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264" is_pre_election: true
I20260928 15:57:03.591988 25223 raft_consensus.cc:2454] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 0.
I20260928 15:57:03.592026 25301 raft_consensus.cc:2781] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:03.592157 25301 raft_consensus.cc:482] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:03.592257 25301 raft_consensus.cc:3037] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:03.594463 25301 raft_consensus.cc:504] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.595070 25301 leader_election.cc:288] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 election: Requested vote from peers c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:03.595559 25223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5497c45900f41fe8ae15f40785d13c2" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264"
I20260928 15:57:03.595571 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5497c45900f41fe8ae15f40785d13c2" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c08d9273a1194d018dad3096c2ff81c0"
I20260928 15:57:03.595786 24951 raft_consensus.cc:3037] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:03.595798 25223 raft_consensus.cc:3037] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:03.598898 25223 raft_consensus.cc:2454] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 1.
I20260928 15:57:03.599114 24951 raft_consensus.cc:2454] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 1.
I20260928 15:57:03.599347 25018 leader_election.cc:302] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45cc58619d14ff0ae01146b8c09e264, e378d8e0acaf45419c16c07c462d8739; no voters:
I20260928 15:57:03.599680 25301 raft_consensus.cc:2781] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:03.600279 25301 raft_consensus.cc:686] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 1 LEADER]: Becoming Leader. State: Replica: e378d8e0acaf45419c16c07c462d8739, State: Running, Role: LEADER
I20260928 15:57:03.600682 25301 consensus_queue.cc:237] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:03.604430 24788 catalog_manager.cc:5495] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 reported cstate change: term changed from 0 to 1, leader changed from <none> to e378d8e0acaf45419c16c07c462d8739 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "e378d8e0acaf45419c16c07c462d8739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:03.632644 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.626196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.22.125.194:42833)
0928 15:57:03.626711 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.626716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.626733 (+ 17us) server_negotiation.cc:407] Connection header received
0928 15:57:03.626805 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.626809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.626886 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.627019 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:03.627489 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.628365 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.630336 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.630736 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.630825 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.631338 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.631943 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.632067 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.632212 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.632279 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":281,"threads_started":1}
I20260928 15:57:03.649511 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.643163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:36745 (local address 127.22.125.194:52459)
0928 15:57:03.643846 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:03.643862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:03.643975 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:03.644398 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:03.644401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:03.644411 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:03.644681 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.644688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.646166 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.646170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.647205 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.647216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.648327 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.648331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.648459 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.648511 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.648961 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.648969 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 15:57:03.649031 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 15:57:03.649132 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":367,"threads_started":1}
I20260928 15:57:03.649493 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.643165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:52459 (local address 127.22.125.193:36745)
0928 15:57:03.643378 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.643384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.643953 (+ 569us) server_negotiation.cc:407] Connection header received
0928 15:57:03.644129 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.644134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.644258 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.644362 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:03.644846 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.646026 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.647350 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.648164 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.648295 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.648680 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.648840 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.648935 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.649079 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.649172 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 15:57:03.658746 25223 raft_consensus.cc:1261] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Refusing update from remote peer e378d8e0acaf45419c16c07c462d8739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:03.659158 24951 raft_consensus.cc:1261] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Refusing update from remote peer e378d8e0acaf45419c16c07c462d8739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:03.659343 25347 consensus_queue.cc:1035] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:03.659878 25301 consensus_queue.cc:1035] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:03.691761 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.686198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:58563 (local address 127.22.125.194:42833)
0928 15:57:03.686348 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:03.686354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:03.686376 (+ 22us) server_negotiation.cc:407] Connection header received
0928 15:57:03.686575 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:03.686579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:03.686672 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:03.686774 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:03.687188 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.688229 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.689800 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.690342 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.690481 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.690883 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.691010 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.691104 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:03.691400 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 15:57:03.691475 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 15:57:03.691811 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:03.686096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:42833 (local address 127.22.125.193:58563)
0928 15:57:03.686317 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:03.686334 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:03.686424 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:03.686810 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:03.686815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:03.686830 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:03.687044 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.687054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.688366 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.688380 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.689618 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:03.689633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:03.690489 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:03.690495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:03.690660 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:03.690723 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:03.691273 (+ 550us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:03.691285 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 15:57:03.691384 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 15:57:03.691528 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 15:57:03.692874 25322 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 15:57:05.284607 24788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:05.284837 24788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:05.288662 24788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Sending DeleteTablet for 3 replicas of tablet d5497c45900f41fe8ae15f40785d13c2
I20260928 15:57:05.289386 24925 tablet_service.cc:1514] Processing DeleteTablet for tablet d5497c45900f41fe8ae15f40785d13c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:05 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:40838
I20260928 15:57:05.289433 25065 tablet_service.cc:1514] Processing DeleteTablet for tablet d5497c45900f41fe8ae15f40785d13c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:05 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:41282
I20260928 15:57:05.289644 25203 tablet_service.cc:1514] Processing DeleteTablet for tablet d5497c45900f41fe8ae15f40785d13c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:05 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:56282
I20260928 15:57:05.290011 25379 tablet_replica.cc:330] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: stopping tablet replica
I20260928 15:57:05.290233 25380 tablet_replica.cc:330] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: stopping tablet replica
I20260928 15:57:05.290266 25379 raft_consensus.cc:2229] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:05.290457 25381 tablet_replica.cc:330] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: stopping tablet replica
I20260928 15:57:05.290491 25380 raft_consensus.cc:2229] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:05.290557 25379 raft_consensus.cc:2258] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:05.290982 25381 raft_consensus.cc:2229] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:05.291281 25381 raft_consensus.cc:2258] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:05.291468 25380 raft_consensus.cc:2258] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:05.292430 25381 ts_tablet_manager.cc:1905] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:05.292480 25380 ts_tablet_manager.cc:1905] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:05.299333 25379 ts_tablet_manager.cc:1905] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:05.299489 25380 ts_tablet_manager.cc:1918] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 15:57:05.299649 25380 log.cc:1196] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/d5497c45900f41fe8ae15f40785d13c2
I20260928 15:57:05.299791 25381 ts_tablet_manager.cc:1918] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 15:57:05.299971 25381 log.cc:1196] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/d5497c45900f41fe8ae15f40785d13c2
I20260928 15:57:05.300184 25380 ts_tablet_manager.cc:1939] T d5497c45900f41fe8ae15f40785d13c2 P e378d8e0acaf45419c16c07c462d8739: Deleting consensus metadata
I20260928 15:57:05.300464 25381 ts_tablet_manager.cc:1939] T d5497c45900f41fe8ae15f40785d13c2 P a45cc58619d14ff0ae01146b8c09e264: Deleting consensus metadata
I20260928 15:57:05.301396 24776 catalog_manager.cc:4841] TS e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833): tablet d5497c45900f41fe8ae15f40785d13c2 (table test-table [id=296fe47e98664af29b01aeb26a5f41fd]) successfully deleted
I20260928 15:57:05.301621 24775 catalog_manager.cc:4841] TS a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547): tablet d5497c45900f41fe8ae15f40785d13c2 (table test-table [id=296fe47e98664af29b01aeb26a5f41fd]) successfully deleted
I20260928 15:57:05.304713 25379 ts_tablet_manager.cc:1918] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 15:57:05.304867 25379 log.cc:1196] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/d5497c45900f41fe8ae15f40785d13c2
I20260928 15:57:05.305253 25379 ts_tablet_manager.cc:1939] T d5497c45900f41fe8ae15f40785d13c2 P c08d9273a1194d018dad3096c2ff81c0: Deleting consensus metadata
I20260928 15:57:05.306185 24776 catalog_manager.cc:4841] TS c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745): tablet d5497c45900f41fe8ae15f40785d13c2 (table test-table [id=296fe47e98664af29b01aeb26a5f41fd]) successfully deleted
Sep 28 15:57:05 dist-test-slave-9cqd krb5kdc[24744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611025, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 15:57:05 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611025, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:05.343410 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:05.325822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.22.125.254:32825)
0928 15:57:05.326399 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:05.326403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:05.326447 (+ 44us) server_negotiation.cc:407] Connection header received
0928 15:57:05.326626 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:05.326637 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:05.326702 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:05.326794 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:05.328035 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:05.328847 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:05.330011 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:05.330399 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:05.330489 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:05.338222 (+ 7733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:05.338227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:05.338230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:05.338263 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:05.341202 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:05.341844 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:05.341852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:05.341856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:05.341989 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:05.342299 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:05.342302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:05.342304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:05.342574 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:05.342752 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:05.342982 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 15:57:05.343143 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":326,"threads_started":1}
I20260928 15:57:05.346751 24788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:05.347440 24788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:05.353822 24925 tablet_service.cc:1467] Processing CreateTablet for tablet 457d7f654bdf4540a8f681fc5983d282 (DEFAULT_TABLE table=test-table [id=51139a61f2914c948f60cea1809bc28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:05.354035 25065 tablet_service.cc:1467] Processing CreateTablet for tablet 457d7f654bdf4540a8f681fc5983d282 (DEFAULT_TABLE table=test-table [id=51139a61f2914c948f60cea1809bc28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:05.354280 24925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 457d7f654bdf4540a8f681fc5983d282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:05.354449 25065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 457d7f654bdf4540a8f681fc5983d282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:05.354415 25203 tablet_service.cc:1467] Processing CreateTablet for tablet 457d7f654bdf4540a8f681fc5983d282 (DEFAULT_TABLE table=test-table [id=51139a61f2914c948f60cea1809bc28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:05.354864 25203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 457d7f654bdf4540a8f681fc5983d282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:05.360380 25390 tablet_bootstrap.cc:492] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: Bootstrap starting.
I20260928 15:57:05.361245 25392 tablet_bootstrap.cc:492] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: Bootstrap starting.
I20260928 15:57:05.362300 25390 tablet_bootstrap.cc:654] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:05.363095 25392 tablet_bootstrap.cc:654] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:05.365204 25390 tablet_bootstrap.cc:492] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: No bootstrap required, opened a new log
I20260928 15:57:05.365393 25390 ts_tablet_manager.cc:1397] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260928 15:57:05.366422 25390 raft_consensus.cc:348] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.366734 25390 raft_consensus.cc:374] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:05.366900 25390 raft_consensus.cc:729] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45cc58619d14ff0ae01146b8c09e264, State: Initialized, Role: FOLLOWER
I20260928 15:57:05.367244 25390 consensus_queue.cc:260] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.368264 25390 ts_tablet_manager.cc:1428] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 15:57:05.368425 25391 tablet_bootstrap.cc:492] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: Bootstrap starting.
I20260928 15:57:05.369555 25392 tablet_bootstrap.cc:492] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: No bootstrap required, opened a new log
I20260928 15:57:05.369722 25392 ts_tablet_manager.cc:1397] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260928 15:57:05.370246 25391 tablet_bootstrap.cc:654] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:05.370636 25392 raft_consensus.cc:348] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.370901 25392 raft_consensus.cc:374] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:05.371001 25392 raft_consensus.cc:729] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e378d8e0acaf45419c16c07c462d8739, State: Initialized, Role: FOLLOWER
I20260928 15:57:05.371302 25392 consensus_queue.cc:260] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.371886 25392 ts_tablet_manager.cc:1428] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260928 15:57:05.373255 25391 tablet_bootstrap.cc:492] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: No bootstrap required, opened a new log
I20260928 15:57:05.373409 25391 ts_tablet_manager.cc:1397] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260928 15:57:05.374125 25391 raft_consensus.cc:348] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.374358 25391 raft_consensus.cc:374] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:05.374464 25391 raft_consensus.cc:729] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Initialized, Role: FOLLOWER
I20260928 15:57:05.374706 25391 consensus_queue.cc:260] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.375317 25391 ts_tablet_manager.cc:1428] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 15:57:05.384976 25304 raft_consensus.cc:482] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:05.385130 25304 raft_consensus.cc:504] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.385744 25304 leader_election.cc:288] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:05.386140 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457d7f654bdf4540a8f681fc5983d282" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e378d8e0acaf45419c16c07c462d8739" is_pre_election: true
I20260928 15:57:05.386224 25223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457d7f654bdf4540a8f681fc5983d282" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264" is_pre_election: true
I20260928 15:57:05.386399 25084 raft_consensus.cc:2454] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 0.
I20260928 15:57:05.386469 25223 raft_consensus.cc:2454] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 0.
I20260928 15:57:05.386866 24879 leader_election.cc:302] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c08d9273a1194d018dad3096c2ff81c0, e378d8e0acaf45419c16c07c462d8739; no voters:
I20260928 15:57:05.387130 25304 raft_consensus.cc:2781] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:05.387229 25304 raft_consensus.cc:482] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:05.387331 25304 raft_consensus.cc:3037] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:05.389340 25304 raft_consensus.cc:504] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.389847 25304 leader_election.cc:288] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 election: Requested vote from peers e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:05.390311 25223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457d7f654bdf4540a8f681fc5983d282" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264"
I20260928 15:57:05.390323 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457d7f654bdf4540a8f681fc5983d282" candidate_uuid: "c08d9273a1194d018dad3096c2ff81c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e378d8e0acaf45419c16c07c462d8739"
I20260928 15:57:05.390527 25084 raft_consensus.cc:3037] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:05.390549 25223 raft_consensus.cc:3037] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:05.393455 25084 raft_consensus.cc:2454] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 1.
I20260928 15:57:05.393455 25223 raft_consensus.cc:2454] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08d9273a1194d018dad3096c2ff81c0 in term 1.
I20260928 15:57:05.393843 24878 leader_election.cc:302] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45cc58619d14ff0ae01146b8c09e264, c08d9273a1194d018dad3096c2ff81c0; no voters:
I20260928 15:57:05.394138 25304 raft_consensus.cc:2781] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:05.394330 25304 raft_consensus.cc:686] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 1 LEADER]: Becoming Leader. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Running, Role: LEADER
I20260928 15:57:05.394623 25304 consensus_queue.cc:237] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.397023 24788 catalog_manager.cc:5495] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193). New cstate: current_term: 1 leader_uuid: "c08d9273a1194d018dad3096c2ff81c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:05.420331 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:05.415698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.22.125.193:36745)
0928 15:57:05.416270 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:05.416274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:05.416293 (+ 19us) server_negotiation.cc:407] Connection header received
0928 15:57:05.416341 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:05.416344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:05.416402 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:05.416484 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:05.416805 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:05.417576 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:05.418779 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:05.419198 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:05.419287 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:05.419673 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:05.419780 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:05.419850 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:05.419988 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 15:57:05.420087 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":340,"threads_started":1}
I20260928 15:57:05.428586 25223 raft_consensus.cc:1261] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Refusing update from remote peer c08d9273a1194d018dad3096c2ff81c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:05.428607 25084 raft_consensus.cc:1261] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Refusing update from remote peer c08d9273a1194d018dad3096c2ff81c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:05.429147 25401 consensus_queue.cc:1035] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:05.429445 25374 consensus_queue.cc:1035] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:05.478247 24787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:05.478472 24787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:05.481351 24787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Sending DeleteTablet for 3 replicas of tablet 457d7f654bdf4540a8f681fc5983d282
I20260928 15:57:05.482153 25065 tablet_service.cc:1514] Processing DeleteTablet for tablet 457d7f654bdf4540a8f681fc5983d282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:05 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:41282
I20260928 15:57:05.482203 25203 tablet_service.cc:1514] Processing DeleteTablet for tablet 457d7f654bdf4540a8f681fc5983d282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:05 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:56282
I20260928 15:57:05.482360 24926 tablet_service.cc:1514] Processing DeleteTablet for tablet 457d7f654bdf4540a8f681fc5983d282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:05 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:40838
I20260928 15:57:05.482448 25380 tablet_replica.cc:330] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: stopping tablet replica
I20260928 15:57:05.482457 25381 tablet_replica.cc:330] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: stopping tablet replica
I20260928 15:57:05.482605 25379 tablet_replica.cc:330] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: stopping tablet replica
I20260928 15:57:05.482632 25381 raft_consensus.cc:2229] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:05.482636 25380 raft_consensus.cc:2229] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:05.482782 25379 raft_consensus.cc:2229] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:05.482793 25381 raft_consensus.cc:2258] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:05.482803 25380 raft_consensus.cc:2258] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:05.483091 25379 raft_consensus.cc:2258] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:05.484119 24787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 15:57:05.484323 25379 ts_tablet_manager.cc:1905] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:05.484349 25381 ts_tablet_manager.cc:1905] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:05.484674 25380 ts_tablet_manager.cc:1905] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 15:57:05.484750 24787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:05.491031 25381 ts_tablet_manager.cc:1918] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:05.491031 25380 ts_tablet_manager.cc:1918] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:05.491195 25380 log.cc:1196] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/457d7f654bdf4540a8f681fc5983d282
I20260928 15:57:05.491385 25381 log.cc:1196] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/457d7f654bdf4540a8f681fc5983d282
I20260928 15:57:05.491865 25381 ts_tablet_manager.cc:1939] T 457d7f654bdf4540a8f681fc5983d282 P a45cc58619d14ff0ae01146b8c09e264: Deleting consensus metadata
I20260928 15:57:05.492069 25379 ts_tablet_manager.cc:1918] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:05.492020 25203 tablet_service.cc:1467] Processing CreateTablet for tablet f41f205d9b7b498f80bf66f2e8f53173 (DEFAULT_TABLE table=test-table [id=4eba4932317542ae9a7102d51e91633f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:05.492273 25379 log.cc:1196] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/457d7f654bdf4540a8f681fc5983d282
I20260928 15:57:05.492501 25203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f41f205d9b7b498f80bf66f2e8f53173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:05.492753 25379 ts_tablet_manager.cc:1939] T 457d7f654bdf4540a8f681fc5983d282 P c08d9273a1194d018dad3096c2ff81c0: Deleting consensus metadata
I20260928 15:57:05.493196 24926 tablet_service.cc:1467] Processing CreateTablet for tablet f41f205d9b7b498f80bf66f2e8f53173 (DEFAULT_TABLE table=test-table [id=4eba4932317542ae9a7102d51e91633f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:05.492461 25065 tablet_service.cc:1467] Processing CreateTablet for tablet f41f205d9b7b498f80bf66f2e8f53173 (DEFAULT_TABLE table=test-table [id=4eba4932317542ae9a7102d51e91633f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:05.493558 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f41f205d9b7b498f80bf66f2e8f53173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:05.493822 25065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f41f205d9b7b498f80bf66f2e8f53173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:05.496312 24776 catalog_manager.cc:4841] TS c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745): tablet 457d7f654bdf4540a8f681fc5983d282 (table test-table [id=51139a61f2914c948f60cea1809bc28e]) successfully deleted
I20260928 15:57:05.491633 25380 ts_tablet_manager.cc:1939] T 457d7f654bdf4540a8f681fc5983d282 P e378d8e0acaf45419c16c07c462d8739: Deleting consensus metadata
I20260928 15:57:05.497872 24776 catalog_manager.cc:4841] TS e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833): tablet 457d7f654bdf4540a8f681fc5983d282 (table test-table [id=51139a61f2914c948f60cea1809bc28e]) successfully deleted
I20260928 15:57:05.498615 25391 tablet_bootstrap.cc:492] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: Bootstrap starting.
I20260928 15:57:05.498790 25390 tablet_bootstrap.cc:492] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: Bootstrap starting.
I20260928 15:57:05.500206 25392 tablet_bootstrap.cc:492] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: Bootstrap starting.
I20260928 15:57:05.500319 25390 tablet_bootstrap.cc:654] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:05.500317 25391 tablet_bootstrap.cc:654] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:05.501820 25392 tablet_bootstrap.cc:654] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:05.503103 24775 catalog_manager.cc:4841] TS a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547): tablet 457d7f654bdf4540a8f681fc5983d282 (table test-table [id=51139a61f2914c948f60cea1809bc28e]) successfully deleted
I20260928 15:57:05.512225 25391 tablet_bootstrap.cc:492] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: No bootstrap required, opened a new log
I20260928 15:57:05.512516 25391 ts_tablet_manager.cc:1397] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260928 15:57:05.513517 25391 raft_consensus.cc:348] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.513760 25391 raft_consensus.cc:374] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:05.513895 25391 raft_consensus.cc:729] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08d9273a1194d018dad3096c2ff81c0, State: Initialized, Role: FOLLOWER
I20260928 15:57:05.514017 25392 tablet_bootstrap.cc:492] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: No bootstrap required, opened a new log
I20260928 15:57:05.514207 25392 ts_tablet_manager.cc:1397] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260928 15:57:05.514197 25391 consensus_queue.cc:260] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.514984 25391 ts_tablet_manager.cc:1428] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:05.515220 25390 tablet_bootstrap.cc:492] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: No bootstrap required, opened a new log
I20260928 15:57:05.514992 25392 raft_consensus.cc:348] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.515429 25390 ts_tablet_manager.cc:1397] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20260928 15:57:05.515461 25392 raft_consensus.cc:374] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:05.515565 25392 raft_consensus.cc:729] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e378d8e0acaf45419c16c07c462d8739, State: Initialized, Role: FOLLOWER
I20260928 15:57:05.515798 25392 consensus_queue.cc:260] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.516131 25390 raft_consensus.cc:348] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.516412 25390 raft_consensus.cc:374] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:05.516516 25390 raft_consensus.cc:729] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45cc58619d14ff0ae01146b8c09e264, State: Initialized, Role: FOLLOWER
I20260928 15:57:05.516527 25392 ts_tablet_manager.cc:1428] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:05.516760 25390 consensus_queue.cc:260] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.517293 25390 ts_tablet_manager.cc:1428] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:05.739893 25357 raft_consensus.cc:482] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:05.740161 25357 raft_consensus.cc:504] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.740947 25357 leader_election.cc:288] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:05.741458 24950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f41f205d9b7b498f80bf66f2e8f53173" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c08d9273a1194d018dad3096c2ff81c0" is_pre_election: true
I20260928 15:57:05.741556 25224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f41f205d9b7b498f80bf66f2e8f53173" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264" is_pre_election: true
I20260928 15:57:05.741684 24950 raft_consensus.cc:2454] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 0.
I20260928 15:57:05.741858 25224 raft_consensus.cc:2454] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 0.
I20260928 15:57:05.742175 25019 leader_election.cc:302] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c08d9273a1194d018dad3096c2ff81c0, e378d8e0acaf45419c16c07c462d8739; no voters:
I20260928 15:57:05.742452 25357 raft_consensus.cc:2781] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:05.742569 25357 raft_consensus.cc:482] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:05.742659 25357 raft_consensus.cc:3037] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:05.744738 25357 raft_consensus.cc:504] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.745308 25357 leader_election.cc:288] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 election: Requested vote from peers c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745), a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547)
I20260928 15:57:05.745676 24950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f41f205d9b7b498f80bf66f2e8f53173" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c08d9273a1194d018dad3096c2ff81c0"
I20260928 15:57:05.745756 25224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f41f205d9b7b498f80bf66f2e8f53173" candidate_uuid: "e378d8e0acaf45419c16c07c462d8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45cc58619d14ff0ae01146b8c09e264"
I20260928 15:57:05.745867 24950 raft_consensus.cc:3037] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:05.745963 25224 raft_consensus.cc:3037] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:05.748428 24950 raft_consensus.cc:2454] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 1.
I20260928 15:57:05.748411 25224 raft_consensus.cc:2454] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e378d8e0acaf45419c16c07c462d8739 in term 1.
I20260928 15:57:05.748884 25018 leader_election.cc:302] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a45cc58619d14ff0ae01146b8c09e264, c08d9273a1194d018dad3096c2ff81c0, e378d8e0acaf45419c16c07c462d8739; no voters:
I20260928 15:57:05.749205 25357 raft_consensus.cc:2781] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:05.749351 25357 raft_consensus.cc:686] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 1 LEADER]: Becoming Leader. State: Replica: e378d8e0acaf45419c16c07c462d8739, State: Running, Role: LEADER
I20260928 15:57:05.749652 25357 consensus_queue.cc:237] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } }
I20260928 15:57:05.752242 24788 catalog_manager.cc:5495] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 reported cstate change: term changed from 0 to 1, leader changed from <none> to e378d8e0acaf45419c16c07c462d8739 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "e378d8e0acaf45419c16c07c462d8739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e378d8e0acaf45419c16c07c462d8739" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:05.811218 25409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:05.805660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41294 (local address 127.22.125.194:42833)
0928 15:57:05.806122 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:05.806128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:05.806158 (+ 30us) server_negotiation.cc:407] Connection header received
0928 15:57:05.806217 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:05.806222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:05.806295 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:05.806455 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:05.806804 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:05.807958 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:05.809320 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:05.809989 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:05.810108 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:05.810489 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:05.810616 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:05.810702 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:05.810825 (+ 123us) server_negotiation.cc:299] Negotiation successful
0928 15:57:05.810906 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":235,"threads_started":1}
I20260928 15:57:05.820489 24950 raft_consensus.cc:1261] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Refusing update from remote peer e378d8e0acaf45419c16c07c462d8739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:05.820500 25224 raft_consensus.cc:1261] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Refusing update from remote peer e378d8e0acaf45419c16c07c462d8739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:05.821029 25357 consensus_queue.cc:1035] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45cc58619d14ff0ae01146b8c09e264" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:05.821313 25373 consensus_queue.cc:1035] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c08d9273a1194d018dad3096c2ff81c0" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 15:57:05.838989 25362 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 15:57:07.418421 24788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:07.418648 24788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:07.422341 24788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a1f53f01f134cf883f222b3884f4316: Sending DeleteTablet for 3 replicas of tablet f41f205d9b7b498f80bf66f2e8f53173
I20260928 15:57:07.423106 24926 tablet_service.cc:1514] Processing DeleteTablet for tablet f41f205d9b7b498f80bf66f2e8f53173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:07 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:40838
I20260928 15:57:07.423228 25065 tablet_service.cc:1514] Processing DeleteTablet for tablet f41f205d9b7b498f80bf66f2e8f53173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:07 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:41282
I20260928 15:57:07.423305 25203 tablet_service.cc:1514] Processing DeleteTablet for tablet f41f205d9b7b498f80bf66f2e8f53173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:07 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:56282
I20260928 15:57:07.423825 25437 tablet_replica.cc:330] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: stopping tablet replica
I20260928 15:57:07.424042 25438 tablet_replica.cc:330] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: stopping tablet replica
I20260928 15:57:07.424077 25437 raft_consensus.cc:2229] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:07.424304 25438 raft_consensus.cc:2229] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:07.424676 25438 raft_consensus.cc:2258] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:07.424786 25437 raft_consensus.cc:2258] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:07.425575 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24860
I20260928 15:57:07.425698 25438 ts_tablet_manager.cc:1905] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:07.426194 25437 ts_tablet_manager.cc:1905] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:07.433698 25438 ts_tablet_manager.cc:1918] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 15:57:07.433923 25438 log.cc:1196] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/f41f205d9b7b498f80bf66f2e8f53173
I20260928 15:57:07.434509 25438 ts_tablet_manager.cc:1939] T f41f205d9b7b498f80bf66f2e8f53173 P e378d8e0acaf45419c16c07c462d8739: Deleting consensus metadata
I20260928 15:57:07.435331 25437 ts_tablet_manager.cc:1918] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 15:57:07.435582 25437 log.cc:1196] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/f41f205d9b7b498f80bf66f2e8f53173
I20260928 15:57:07.435830 24776 catalog_manager.cc:4841] TS e378d8e0acaf45419c16c07c462d8739 (127.22.125.194:42833): tablet f41f205d9b7b498f80bf66f2e8f53173 (table test-table [id=4eba4932317542ae9a7102d51e91633f]) successfully deleted
I20260928 15:57:07.436269 25437 ts_tablet_manager.cc:1939] T f41f205d9b7b498f80bf66f2e8f53173 P c08d9273a1194d018dad3096c2ff81c0: Deleting consensus metadata
I20260928 15:57:07.436466 25439 tablet_replica.cc:330] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: stopping tablet replica
I20260928 15:57:07.436776 25439 raft_consensus.cc:2229] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:07.437105 25439 raft_consensus.cc:2258] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:07.437573 24776 catalog_manager.cc:4841] TS c08d9273a1194d018dad3096c2ff81c0 (127.22.125.193:36745): tablet f41f205d9b7b498f80bf66f2e8f53173 (table test-table [id=4eba4932317542ae9a7102d51e91633f]) successfully deleted
I20260928 15:57:07.438109 25439 ts_tablet_manager.cc:1905] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:07.444151 25439 ts_tablet_manager.cc:1918] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 15:57:07.444355 25439 log.cc:1196] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/f41f205d9b7b498f80bf66f2e8f53173
I20260928 15:57:07.444828 25439 ts_tablet_manager.cc:1939] T f41f205d9b7b498f80bf66f2e8f53173 P a45cc58619d14ff0ae01146b8c09e264: Deleting consensus metadata
I20260928 15:57:07.445856 24775 catalog_manager.cc:4841] TS a45cc58619d14ff0ae01146b8c09e264 (127.22.125.195:41547): tablet f41f205d9b7b498f80bf66f2e8f53173 (table test-table [id=4eba4932317542ae9a7102d51e91633f]) successfully deleted
Sep 28 15:57:07 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611025, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:07.455307 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:07.427113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.22.125.193:36745)
0928 15:57:07.437352 (+ 10239us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:07.437358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:07.437377 (+ 19us) server_negotiation.cc:407] Connection header received
0928 15:57:07.437434 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:07.437439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:07.437530 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:07.437645 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:07.439090 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:07.440214 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:07.442128 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:07.442839 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:07.442948 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:07.448285 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:07.448304 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:07.448325 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:07.448363 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:07.452334 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:07.453112 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:07.453122 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:07.453125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:07.453288 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:07.453669 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:07.453674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:07.453677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:07.454191 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:07.454481 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:07.454716 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 15:57:07.454942 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10064,"thread_start_us":9936,"threads_started":1}
W20260928 15:57:07.456305 24991 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40864
W20260928 15:57:07.456914 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24860: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:57:07.457130 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 24860
I20260928 15:57:07.478416 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25000
Sep 28 15:57:07 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611025, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:57:07.492853 25442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:07.479082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41298 (local address 127.22.125.194:42833)
0928 15:57:07.479561 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:07.479566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:07.479581 (+ 15us) server_negotiation.cc:407] Connection header received
0928 15:57:07.479632 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:07.479635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:07.479695 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:07.479786 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:07.481097 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:07.481834 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:07.482966 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:07.483344 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:07.483432 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:07.487585 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:07.487598 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:07.487612 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:07.487644 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:07.490286 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:07.490890 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:07.490898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:07.490900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:07.491031 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:07.491392 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:07.491396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:07.491398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:07.491743 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:07.491968 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:07.492311 (+ 343us) server_negotiation.cc:299] Negotiation successful
0928 15:57:07.492473 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":271,"threads_started":1}
W20260928 15:57:07.493548 25130 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41298
W20260928 15:57:07.494159 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25000: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:57:07.494338 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 25000
I20260928 15:57:07.517060 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25139
Sep 28 15:57:07 dist-test-slave-9cqd krb5kdc[24744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611025, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:57:07.532693 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:07.517830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56286 (local address 127.22.125.195:41547)
0928 15:57:07.518341 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:07.518346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:07.518361 (+ 15us) server_negotiation.cc:407] Connection header received
0928 15:57:07.518416 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:07.518420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:07.518480 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:07.518568 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:07.519761 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:07.520539 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:07.521830 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:07.522259 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:07.522353 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:07.526832 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:07.526846 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:07.526862 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:07.526906 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:07.529948 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:07.530629 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:07.530637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:07.530639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:07.530771 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:07.531143 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:07.531149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:07.531152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:07.531504 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:07.531804 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:07.532141 (+ 337us) server_negotiation.cc:299] Negotiation successful
0928 15:57:07.532372 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":272,"threads_started":1}
W20260928 15:57:07.533341 25269 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56286
W20260928 15:57:07.533938 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25139: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:57:07.534109 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 25139
I20260928 15:57:07.552239 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 24757
I20260928 15:57:07.553437 24829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:07.669976 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 24757
2026-09-28T15:57:07Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8555 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:07 dist-test-slave-9cqd krb5kdc[25447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:07 dist-test-slave-9cqd krb5kdc[25447](info): set up 2 sockets
Sep 28 15:57:07 dist-test-slave-9cqd krb5kdc[25447](info): commencing operation
krb5kdc: starting...
W20260928 15:57:08.742884 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:08 dist-test-slave-9cqd krb5kdc[25447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:08.931782 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34693
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:34693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:09.248474 25460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:09.248801 25460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:09.248974 25460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:09.256130 25460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:09.256273 25460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:09.256392 25460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:09.256493 25460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:09.256569 25460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:09.266229 25460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:34693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:09.267547 25460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:09.268911 25460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:09.280946 25466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:09.281020 25465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:09.281235 25460 server_base.cc:1034] running on GCE node
W20260928 15:57:09.280984 25468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:09.282141 25460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:09.283432 25460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:09.284631 25460 hybrid_clock.cc:648] HybridClock initialized: now 1790611029284578 us; error 39 us; skew 500 ppm
I20260928 15:57:09.285003 25460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:09 dist-test-slave-9cqd krb5kdc[25447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611029, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:09.288107 25460 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:09.289487 25460 webserver.cc:460] Webserver started at http://127.22.125.254:39655/ using document root <none> and password file <none>
I20260928 15:57:09.290158 25460 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:09.290301 25460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:09.290607 25460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:09.293522 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "188fbc858b4a46cd8751c00ffee71e23"
format_stamp: "Formatted at 2026-09-28 15:57:09 on dist-test-slave-9cqd"
server_key: "2e5c546008bd91613c81e46076efdb7c"
server_key_iv: "ddf307e14234b94db651074147ab6044"
server_key_version: "encryptionkey@0"
I20260928 15:57:09.294260 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "188fbc858b4a46cd8751c00ffee71e23"
format_stamp: "Formatted at 2026-09-28 15:57:09 on dist-test-slave-9cqd"
server_key: "2e5c546008bd91613c81e46076efdb7c"
server_key_iv: "ddf307e14234b94db651074147ab6044"
server_key_version: "encryptionkey@0"
I20260928 15:57:09.299147 25460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260928 15:57:09.302496 25475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:09.303697 25460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 15:57:09.303877 25460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "188fbc858b4a46cd8751c00ffee71e23"
format_stamp: "Formatted at 2026-09-28 15:57:09 on dist-test-slave-9cqd"
server_key: "2e5c546008bd91613c81e46076efdb7c"
server_key_iv: "ddf307e14234b94db651074147ab6044"
server_key_version: "encryptionkey@0"
I20260928 15:57:09.304100 25460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:09.360852 25460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:09.364794 25460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:09.365111 25460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:09.383714 25460 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:34693
I20260928 15:57:09.383735 25526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:34693 every 8 connection(s)
I20260928 15:57:09.385082 25460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:09.388785 25527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:09.389420 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 25460
I20260928 15:57:09.389691 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:09.390491 23031 external_mini_cluster.cc:1428] Setting key 04767e4a2297bb4b16abce4a5cc5f156
I20260928 15:57:09.401114 25527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23: Bootstrap starting.
Sep 28 15:57:09 dist-test-slave-9cqd krb5kdc[25447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:09.404505 25527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:09.405529 25527 log.cc:824] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:09.408720 25527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23: No bootstrap required, opened a new log
I20260928 15:57:09.412249 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:09.392649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.22.125.254:34693)
0928 15:57:09.393406 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:09.393418 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:09.393463 (+ 45us) server_negotiation.cc:407] Connection header received
0928 15:57:09.395260 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:09.395284 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:09.395674 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:09.396085 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:09.397512 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:09.398794 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:09.400011 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:09.400709 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:09.400927 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:09.405043 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:09.405051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:09.405064 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:09.405097 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:09.408286 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:09.409027 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:09.409034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:09.409040 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:09.409211 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:09.409597 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:09.409602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:09.409604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:09.410071 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:09.410430 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:09.410826 (+ 396us) server_negotiation.cc:299] Negotiation successful
0928 15:57:09.411231 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":334,"threads_started":1}
I20260928 15:57:09.416553 25527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188fbc858b4a46cd8751c00ffee71e23" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34693 } }
I20260928 15:57:09.416886 25527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:09.417006 25527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188fbc858b4a46cd8751c00ffee71e23, State: Initialized, Role: FOLLOWER
I20260928 15:57:09.417616 25527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188fbc858b4a46cd8751c00ffee71e23" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34693 } }
I20260928 15:57:09.417855 25527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:09.417997 25527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:09.418171 25527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:09.420203 25527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188fbc858b4a46cd8751c00ffee71e23" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34693 } }
I20260928 15:57:09.420663 25527 leader_election.cc:302] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 188fbc858b4a46cd8751c00ffee71e23; no voters:
I20260928 15:57:09.421188 25527 leader_election.cc:288] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:09.421361 25532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:09.422051 25532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [term 1 LEADER]: Becoming Leader. State: Replica: 188fbc858b4a46cd8751c00ffee71e23, State: Running, Role: LEADER
I20260928 15:57:09.422530 25532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188fbc858b4a46cd8751c00ffee71e23" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34693 } }
I20260928 15:57:09.423346 25527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:09.425503 25533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "188fbc858b4a46cd8751c00ffee71e23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188fbc858b4a46cd8751c00ffee71e23" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34693 } } }
I20260928 15:57:09.425590 25534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 188fbc858b4a46cd8751c00ffee71e23. Latest consensus state: current_term: 1 leader_uuid: "188fbc858b4a46cd8751c00ffee71e23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188fbc858b4a46cd8751c00ffee71e23" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34693 } } }
I20260928 15:57:09.425982 25533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:09.426122 25534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:09.427361 25538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:09.433414 25538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:09.442487 25538 catalog_manager.cc:1347] Generated new cluster ID: 45648d854fd1487dba63cfe87e24c2b7
I20260928 15:57:09.442595 25538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:09.465903 25538 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:09.467700 25538 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:09.479914 25538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 188fbc858b4a46cd8751c00ffee71e23: Generated new TSK 0
I20260928 15:57:09.480914 25538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:57:09.536695 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34693
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:09.861888 25553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:09.862147 25553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:09.862249 25553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:09.869412 25553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:09.869503 25553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:09.869606 25553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:09.879086 25553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:09.880244 25553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:09.881446 25553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:09.894358 25559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:09.894393 25561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:09.894425 25558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:09.894785 25553 server_base.cc:1034] running on GCE node
I20260928 15:57:09.895599 25553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:09.896522 25553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:09.897727 25553 hybrid_clock.cc:648] HybridClock initialized: now 1790611029897664 us; error 50 us; skew 500 ppm
I20260928 15:57:09.898062 25553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:09 dist-test-slave-9cqd krb5kdc[25447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611029, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:09.901119 25553 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:57:09.902631 25553 webserver.cc:460] Webserver started at http://127.22.125.193:38939/ using document root <none> and password file <none>
I20260928 15:57:09.903318 25553 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:09.903466 25553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:09.903765 25553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:09.906633 25553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "d75b7b7f2f844c3bb085fa76fa11d674"
format_stamp: "Formatted at 2026-09-28 15:57:09 on dist-test-slave-9cqd"
server_key: "bc8c35bfae85b51570f0c3c5c70e7fc0"
server_key_iv: "975e3551311c1b75f07d61be9876366d"
server_key_version: "encryptionkey@0"
I20260928 15:57:09.907490 25553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "d75b7b7f2f844c3bb085fa76fa11d674"
format_stamp: "Formatted at 2026-09-28 15:57:09 on dist-test-slave-9cqd"
server_key: "bc8c35bfae85b51570f0c3c5c70e7fc0"
server_key_iv: "975e3551311c1b75f07d61be9876366d"
server_key_version: "encryptionkey@0"
I20260928 15:57:09.912946 25553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 15:57:09.916251 25568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:09.917492 25553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:09.917668 25553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "d75b7b7f2f844c3bb085fa76fa11d674"
format_stamp: "Formatted at 2026-09-28 15:57:09 on dist-test-slave-9cqd"
server_key: "bc8c35bfae85b51570f0c3c5c70e7fc0"
server_key_iv: "975e3551311c1b75f07d61be9876366d"
server_key_version: "encryptionkey@0"
I20260928 15:57:09.917862 25553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:09.944749 25553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:09.948647 25553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:09.948935 25553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:09.949822 25553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:09.951092 25553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:09.951197 25553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:09.951342 25553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:09.951418 25553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:09.983753 25553 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:35413
I20260928 15:57:09.983834 25680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:35413 every 8 connection(s)
I20260928 15:57:09.985042 25553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:09.993860 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 25553
I20260928 15:57:09.994071 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:09.994664 23031 external_mini_cluster.cc:1428] Setting key 96a61f9584af9f3f5adae9efed2455ea
Sep 28 15:57:09 dist-test-slave-9cqd krb5kdc[25447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611029, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:10.004153 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:09.987523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:57137 (local address 127.22.125.254:34693)
0928 15:57:09.987938 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:09.987942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:09.988965 (+ 1023us) server_negotiation.cc:407] Connection header received
0928 15:57:09.990750 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:09.990755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:09.990846 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:09.990964 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:09.992459 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:09.993169 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:09.994794 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:09.995215 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:09.995347 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:09.998692 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:09.998697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:09.998700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:09.998730 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:10.001214 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:10.001917 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:10.001930 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:10.001933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:10.002051 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:10.002471 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:10.002476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:10.002479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:10.002735 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:10.002892 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:10.003547 (+ 655us) server_negotiation.cc:299] Negotiation successful
0928 15:57:10.003777 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":223,"threads_started":1}
I20260928 15:57:10.005051 25684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:09.988153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34693 (local address 127.22.125.193:57137)
0928 15:57:09.988804 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:09.988846 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:09.990487 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:09.991192 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:09.991200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:09.991683 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:09.992294 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:09.992312 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:09.993327 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:09.993339 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:09.994635 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:09.994647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:09.995443 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:09.995451 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:09.995719 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:09.996430 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:09.996455 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:09.998483 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:10.001354 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:10.001363 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:10.001378 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:10.001785 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:10.002163 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:10.002168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:10.002171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:10.002347 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:10.002871 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:10.002879 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:10.003191 (+ 312us) client_negotiation.cc:769] Sending connection context
0928 15:57:10.003438 (+ 247us) client_negotiation.cc:240] Negotiation successful
0928 15:57:10.003938 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":253,"threads_started":1}
I20260928 15:57:10.006599 25681 heartbeater.cc:347] Connected to a master server at 127.22.125.254:34693
I20260928 15:57:10.006903 25681 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:10.007629 25681 heartbeater.cc:511] Master 127.22.125.254:34693 requested a full tablet report, sending...
I20260928 15:57:10.009752 25492 ts_manager.cc:194] Registered new tserver with Master: d75b7b7f2f844c3bb085fa76fa11d674 (127.22.125.193:35413)
I20260928 15:57:10.011155 25492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:57137
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:57:10.034837 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34693
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:10.348918 25687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:10.349180 25687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:10.349335 25687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:10.356415 25687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:10.356557 25687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:10.356741 25687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:10.365959 25687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:10.367198 25687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:10.368573 25687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:10.381582 25693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:10.381796 25695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:10.382048 25692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:10.382377 25687 server_base.cc:1034] running on GCE node
I20260928 15:57:10.382977 25687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:10.383900 25687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:10.385125 25687 hybrid_clock.cc:648] HybridClock initialized: now 1790611030385087 us; error 55 us; skew 500 ppm
I20260928 15:57:10.385458 25687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:10 dist-test-slave-9cqd krb5kdc[25447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611030, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:10.397851 25687 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:57:10.399279 25687 webserver.cc:460] Webserver started at http://127.22.125.194:37635/ using document root <none> and password file <none>
I20260928 15:57:10.400020 25687 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:10.400211 25687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:10.400528 25687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:10.403360 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "776c9d5788aa42188859f6914caf1194"
format_stamp: "Formatted at 2026-09-28 15:57:10 on dist-test-slave-9cqd"
server_key: "d5cba3f58e8b2f2ec718d144d6ffc602"
server_key_iv: "0c6593ba3c6303e324555e27375dc7de"
server_key_version: "encryptionkey@0"
I20260928 15:57:10.404201 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "776c9d5788aa42188859f6914caf1194"
format_stamp: "Formatted at 2026-09-28 15:57:10 on dist-test-slave-9cqd"
server_key: "d5cba3f58e8b2f2ec718d144d6ffc602"
server_key_iv: "0c6593ba3c6303e324555e27375dc7de"
server_key_version: "encryptionkey@0"
I20260928 15:57:10.409765 25687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260928 15:57:10.413213 25702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:10.414510 25687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260928 15:57:10.414677 25687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "776c9d5788aa42188859f6914caf1194"
format_stamp: "Formatted at 2026-09-28 15:57:10 on dist-test-slave-9cqd"
server_key: "d5cba3f58e8b2f2ec718d144d6ffc602"
server_key_iv: "0c6593ba3c6303e324555e27375dc7de"
server_key_version: "encryptionkey@0"
I20260928 15:57:10.414881 25687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:10.430706 25687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:10.434482 25687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:10.434773 25687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:10.435690 25687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:10.436982 25687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:10.437086 25687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:10.437240 25687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:10.437314 25687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:10.469776 25687 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:35149
I20260928 15:57:10.469820 25814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:35149 every 8 connection(s)
I20260928 15:57:10.471060 25687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:10.471392 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 25687
I20260928 15:57:10.471593 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:10.472175 23031 external_mini_cluster.cc:1428] Setting key ffe189dfa4a10504ed32fb6efcd5ec28
Sep 28 15:57:10 dist-test-slave-9cqd krb5kdc[25447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611030, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 15:57:10.493017 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:10.473868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:37267 (local address 127.22.125.254:34693)
0928 15:57:10.474034 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:10.474038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:10.475437 (+ 1399us) server_negotiation.cc:407] Connection header received
0928 15:57:10.477273 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:10.477278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:10.477371 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:10.477481 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:10.478949 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.479676 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.482362 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.482766 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.482870 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:10.486966 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:10.486970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:10.486973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:10.487003 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:10.489812 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:10.490513 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:10.490523 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:10.490525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:10.490641 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:10.491125 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:10.491132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:10.491139 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:10.491414 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:10.491569 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:10.492433 (+ 864us) server_negotiation.cc:299] Negotiation successful
0928 15:57:10.492660 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
I20260928 15:57:10.494095 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:10.474547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34693 (local address 127.22.125.194:37267)
0928 15:57:10.475289 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:10.475325 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:10.477017 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:10.477717 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:10.477725 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:10.478223 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:10.478779 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:10.478796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.479845 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.479849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:10.482191 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:10.482210 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.482995 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.483004 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:10.483355 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:10.484109 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:10.484143 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:10.486741 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:10.489954 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:10.489963 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:10.489979 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:10.490386 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:10.490751 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:10.490757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:10.490760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:10.490955 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:10.491543 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:10.491551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:10.492000 (+ 449us) client_negotiation.cc:769] Sending connection context
0928 15:57:10.492330 (+ 330us) client_negotiation.cc:240] Negotiation successful
0928 15:57:10.492920 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":315,"threads_started":1}
I20260928 15:57:10.495565 25815 heartbeater.cc:347] Connected to a master server at 127.22.125.254:34693
I20260928 15:57:10.495893 25815 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:10.496814 25815 heartbeater.cc:511] Master 127.22.125.254:34693 requested a full tablet report, sending...
I20260928 15:57:10.498485 25492 ts_manager.cc:194] Registered new tserver with Master: 776c9d5788aa42188859f6914caf1194 (127.22.125.194:35149)
I20260928 15:57:10.499658 25492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:37267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:57:10.515347 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34693
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:10.838195 25820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:10.838478 25820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:10.838631 25820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:10.846266 25820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:10.846383 25820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:10.846563 25820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:10.856396 25820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:10.857692 25820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:10.859010 25820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:10.872350 25825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:10.872395 25826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:10.872443 25828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:10.877512 25820 server_base.cc:1034] running on GCE node
I20260928 15:57:10.878088 25820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:10.879002 25820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:10.880221 25820 hybrid_clock.cc:648] HybridClock initialized: now 1790611030880175 us; error 35 us; skew 500 ppm
I20260928 15:57:10.880530 25820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:10 dist-test-slave-9cqd krb5kdc[25447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611030, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:10.883636 25820 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:57:10.885200 25820 webserver.cc:460] Webserver started at http://127.22.125.195:38023/ using document root <none> and password file <none>
I20260928 15:57:10.885877 25820 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:10.885974 25820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:10.886242 25820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:10.889093 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "746a7a4c3f934b30b78647ce5c955e43"
format_stamp: "Formatted at 2026-09-28 15:57:10 on dist-test-slave-9cqd"
server_key: "33daf971192cf0cae5da0395a797aef8"
server_key_iv: "71408502b1b7d3f8ccd62bbfe043e0bb"
server_key_version: "encryptionkey@0"
I20260928 15:57:10.889791 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "746a7a4c3f934b30b78647ce5c955e43"
format_stamp: "Formatted at 2026-09-28 15:57:10 on dist-test-slave-9cqd"
server_key: "33daf971192cf0cae5da0395a797aef8"
server_key_iv: "71408502b1b7d3f8ccd62bbfe043e0bb"
server_key_version: "encryptionkey@0"
I20260928 15:57:10.894743 25820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 15:57:10.898094 25835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:10.899390 25820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 15:57:10.899566 25820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "746a7a4c3f934b30b78647ce5c955e43"
format_stamp: "Formatted at 2026-09-28 15:57:10 on dist-test-slave-9cqd"
server_key: "33daf971192cf0cae5da0395a797aef8"
server_key_iv: "71408502b1b7d3f8ccd62bbfe043e0bb"
server_key_version: "encryptionkey@0"
I20260928 15:57:10.899766 25820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:10.917927 25820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:10.922410 25820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:10.922701 25820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:10.923681 25820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:10.925238 25820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:10.925339 25820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:10.925489 25820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:10.925559 25820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:10.958910 25820 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:36929
I20260928 15:57:10.958956 25947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:36929 every 8 connection(s)
I20260928 15:57:10.960232 25820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:10.965237 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 25820
I20260928 15:57:10.965456 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:10.965893 23031 external_mini_cluster.cc:1428] Setting key 19f0d35b3306dae0cff029bf8dbd84d2
Sep 28 15:57:10 dist-test-slave-9cqd krb5kdc[25447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611030, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:10.980273 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:10.962948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:59433 (local address 127.22.125.254:34693)
0928 15:57:10.963137 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:10.963143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:10.964394 (+ 1251us) server_negotiation.cc:407] Connection header received
0928 15:57:10.966395 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:10.966401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:10.966499 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:10.966624 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:10.968570 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.969436 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.970928 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.971332 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.971426 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:10.974733 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:10.974737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:10.974740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:10.974772 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:10.977446 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:10.978056 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:10.978059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:10.978062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:10.978187 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:10.978547 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:10.978550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:10.978552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:10.978805 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:10.978912 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:10.979625 (+ 713us) server_negotiation.cc:299] Negotiation successful
0928 15:57:10.979842 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 15:57:10.981280 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:10.963433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34693 (local address 127.22.125.195:59433)
0928 15:57:10.964235 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:10.964279 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:10.966114 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:10.966903 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:10.966914 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:10.967580 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:10.968400 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:10.968425 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.969585 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.969589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:10.970799 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:10.970811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:10.971510 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:10.971514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:10.971702 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:10.972376 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:10.972402 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:10.974522 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:10.977562 (+ 3040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:10.977568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:10.977579 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:10.977951 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:10.978272 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:10.978275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:10.978277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:10.978452 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:10.978900 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:10.978905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:10.979196 (+ 291us) client_negotiation.cc:769] Sending connection context
0928 15:57:10.979476 (+ 280us) client_negotiation.cc:240] Negotiation successful
0928 15:57:10.979967 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":269,"threads_started":1}
I20260928 15:57:10.982559 25948 heartbeater.cc:347] Connected to a master server at 127.22.125.254:34693
I20260928 15:57:10.982873 25948 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:10.983669 25948 heartbeater.cc:511] Master 127.22.125.254:34693 requested a full tablet report, sending...
I20260928 15:57:10.985193 25492 ts_manager.cc:194] Registered new tserver with Master: 746a7a4c3f934b30b78647ce5c955e43 (127.22.125.195:36929)
I20260928 15:57:10.986306 25492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:59433
I20260928 15:57:10.993327 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:57:11.014478 25681 heartbeater.cc:503] Master 127.22.125.254:34693 was elected leader, sending a full tablet report...
W20260928 15:57:11.023854 25683 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:11.021270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.22.125.254:34693)
0928 15:57:11.021482 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:11.021491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:11.021894 (+ 403us) server_negotiation.cc:407] Connection header received
0928 15:57:11.022140 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:11.022152 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:11.022217 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:11.022326 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:11.023657 (+ 1331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42274: BlockingRecv error: recv got EOF from 127.0.0.1:42274 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 15:57:11.027235 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25553
W20260928 15:57:11.029982 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25553: Not authorized: Client connection negotiation failed: client connection to 127.22.125.193:35413: server requires authentication, but client does not have Kerberos credentials available
I20260928 15:57:11.030169 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 25553
I20260928 15:57:11.045368 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25687
W20260928 15:57:11.047850 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25687: Not authorized: Client connection negotiation failed: client connection to 127.22.125.194:35149: server requires authentication, but client does not have Kerberos credentials available
I20260928 15:57:11.048027 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 25687
I20260928 15:57:11.063323 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25820
W20260928 15:57:11.065897 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25820: Not authorized: Client connection negotiation failed: client connection to 127.22.125.195:36929: server requires authentication, but client does not have Kerberos credentials available
I20260928 15:57:11.066039 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 25820
I20260928 15:57:11.081063 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25460
I20260928 15:57:11.082188 25522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:11.185302 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 25460
2026-09-28T15:57:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3512 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:11 dist-test-slave-9cqd krb5kdc[25965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:11 dist-test-slave-9cqd krb5kdc[25965](info): set up 2 sockets
Sep 28 15:57:11 dist-test-slave-9cqd krb5kdc[25965](info): commencing operation
krb5kdc: starting...
W20260928 15:57:12.347375 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.004s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260928 15:57:12.549156 23031 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.202s user 0.001s sys 0.005s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260928 15:57:12.655602 23031 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.106s user 0.000s sys 0.004s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260928 15:57:12.884582 23031 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.229s user 0.002s sys 0.003s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/test-user.keytab.
W20260928 15:57:13.686779 23031 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.802s user 0.000s sys 0.005s
Sep 28 15:57:13 dist-test-slave-9cqd krb5kdc[25965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
W20260928 15:57:13.925997 23031 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.125.254: real 0.189s user 0.000s sys 0.008s
I20260928 15:57:13.926972 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:36445
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:32797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:36445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:14.243566 25978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:14.243844 25978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:14.243958 25978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:14.251158 25978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:14.251253 25978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:14.251387 25978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:14.251508 25978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:14.251596 25978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:14.261269 25978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:32797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:36445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:36445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:14.262506 25978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:14.263744 25978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:14.275707 25984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:14.275646 25986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:14.275748 25983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:14.276403 25978 server_base.cc:1034] running on GCE node
I20260928 15:57:14.277171 25978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:14.278391 25978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:14.279596 25978 hybrid_clock.cc:648] HybridClock initialized: now 1790611034279545 us; error 37 us; skew 500 ppm
I20260928 15:57:14.279954 25978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:14 dist-test-slave-9cqd krb5kdc[25965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611034, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:14.282985 25978 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:14.284461 25978 webserver.cc:460] Webserver started at http://127.22.125.254:40249/ using document root <none> and password file <none>
I20260928 15:57:14.285091 25978 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:14.285192 25978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:14.285445 25978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:14.288522 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "ba914dbc08c740e89503d3f3a788d62d"
format_stamp: "Formatted at 2026-09-28 15:57:14 on dist-test-slave-9cqd"
server_key: "5266c8ea8740a2306c3dd304e1280ab0"
server_key_iv: "62d1c2a9d4ab817d5cd67d2180ba4b54"
server_key_version: "encryptionkey@0"
I20260928 15:57:14.289319 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "ba914dbc08c740e89503d3f3a788d62d"
format_stamp: "Formatted at 2026-09-28 15:57:14 on dist-test-slave-9cqd"
server_key: "5266c8ea8740a2306c3dd304e1280ab0"
server_key_iv: "62d1c2a9d4ab817d5cd67d2180ba4b54"
server_key_version: "encryptionkey@0"
I20260928 15:57:14.294373 25978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:14.297614 25993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:14.298849 25978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:57:14.299029 25978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "ba914dbc08c740e89503d3f3a788d62d"
format_stamp: "Formatted at 2026-09-28 15:57:14 on dist-test-slave-9cqd"
server_key: "5266c8ea8740a2306c3dd304e1280ab0"
server_key_iv: "62d1c2a9d4ab817d5cd67d2180ba4b54"
server_key_version: "encryptionkey@0"
I20260928 15:57:14.299218 25978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:14.319223 25978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:14.323611 25978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:14.323899 25978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:14.341780 25978 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:36445
I20260928 15:57:14.341848 26044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:36445 every 8 connection(s)
I20260928 15:57:14.343075 25978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:14.346740 26045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:14.353334 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 25978
I20260928 15:57:14.353564 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:14.354228 23031 external_mini_cluster.cc:1428] Setting key 784ce2c0ad6a881a4617f92ecb02209a
I20260928 15:57:14.356999 26045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d: Bootstrap starting.
I20260928 15:57:14.360375 26045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:14.361683 26045 log.cc:824] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:14.364759 26045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d: No bootstrap required, opened a new log
Sep 28 15:57:14 dist-test-slave-9cqd krb5kdc[25965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:14.375317 26045 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba914dbc08c740e89503d3f3a788d62d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36445 } }
I20260928 15:57:14.375778 26045 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:14.375898 26045 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba914dbc08c740e89503d3f3a788d62d, State: Initialized, Role: FOLLOWER
I20260928 15:57:14.376606 26045 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba914dbc08c740e89503d3f3a788d62d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36445 } }
I20260928 15:57:14.376924 26045 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:14.377077 26045 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:14.377261 26045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:14.378826 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:14.355996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.22.125.254:36445)
0928 15:57:14.356809 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:14.356821 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:14.356861 (+ 40us) server_negotiation.cc:407] Connection header received
0928 15:57:14.358564 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:14.358584 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:14.358948 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:14.359238 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:14.361201 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:14.362950 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:14.364720 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:14.365439 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:14.365563 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:14.370025 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:14.370035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:14.370053 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:14.370095 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:14.374140 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:14.374787 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:14.374795 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:14.374804 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:14.375020 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:14.375388 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:14.375393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:14.375396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:14.376143 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:14.376458 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:14.376970 (+ 512us) server_negotiation.cc:299] Negotiation successful
0928 15:57:14.377451 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":342,"threads_started":1}
I20260928 15:57:14.380158 26045 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba914dbc08c740e89503d3f3a788d62d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36445 } }
I20260928 15:57:14.380640 26045 leader_election.cc:302] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba914dbc08c740e89503d3f3a788d62d; no voters:
I20260928 15:57:14.381278 26045 leader_election.cc:288] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:14.381407 26050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:14.382735 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 15:57:14.382688 26050 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [term 1 LEADER]: Becoming Leader. State: Replica: ba914dbc08c740e89503d3f3a788d62d, State: Running, Role: LEADER
I20260928 15:57:14.383198 26050 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba914dbc08c740e89503d3f3a788d62d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36445 } }
I20260928 15:57:14.383665 26045 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:14.386222 26051 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba914dbc08c740e89503d3f3a788d62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba914dbc08c740e89503d3f3a788d62d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36445 } } }
I20260928 15:57:14.386665 26051 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:14.387729 26052 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba914dbc08c740e89503d3f3a788d62d. Latest consensus state: current_term: 1 leader_uuid: "ba914dbc08c740e89503d3f3a788d62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba914dbc08c740e89503d3f3a788d62d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36445 } } }
I20260928 15:57:14.388134 26052 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:14.391710 26055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:14.396833 26055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:14.412902 26055 catalog_manager.cc:1347] Generated new cluster ID: 9bcd1ec3c5f24016b9aa1187ba201c73
I20260928 15:57:14.413113 26055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:14.448536 26055 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:14.451139 26055 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:14.459681 26055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba914dbc08c740e89503d3f3a788d62d: Generated new TSK 0
I20260928 15:57:14.460728 26055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 15:57:14.461501 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:14.451549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56862 (local address 127.22.125.254:36445)
0928 15:57:14.451718 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:14.451724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:14.452391 (+ 667us) server_negotiation.cc:407] Connection header received
0928 15:57:14.452597 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:14.452603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:14.452691 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:14.452803 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:14.453990 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:14.455184 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:14.457072 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:14.457793 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:14.457947 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:14.459737 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:14.459741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:14.459744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:14.460615 (+ 871us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:14.460676 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:14.460704 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:14.460834 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:14.461028 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 15:57:14.461129 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 15:57:14.469991 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 25978
I20260928 15:57:14.471338 26040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:14.565850 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 25978
2026-09-28T15:57:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3375 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:14 dist-test-slave-9cqd krb5kdc[26079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:14 dist-test-slave-9cqd krb5kdc[26079](info): set up 2 sockets
Sep 28 15:57:14 dist-test-slave-9cqd krb5kdc[26079](info): commencing operation
krb5kdc: starting...
W20260928 15:57:15.666291 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:15 dist-test-slave-9cqd krb5kdc[26079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:15.849282 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:36341
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:36341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:16.156350 26092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:16.156615 26092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:16.156705 26092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:16.163763 26092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:16.163861 26092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:16.163918 26092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:16.163972 26092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:16.164016 26092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:16.173590 26092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:36341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:36341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:16.174762 26092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:16.175927 26092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:16.187449 26097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:16.187528 26100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:16.187407 26098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:16.188169 26092 server_base.cc:1034] running on GCE node
I20260928 15:57:16.188894 26092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:16.190234 26092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:16.191430 26092 hybrid_clock.cc:648] HybridClock initialized: now 1790611036191392 us; error 36 us; skew 500 ppm
I20260928 15:57:16.191740 26092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:16 dist-test-slave-9cqd krb5kdc[26079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611036, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:16.198618 26092 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:16.200207 26092 webserver.cc:460] Webserver started at http://127.22.125.254:36189/ using document root <none> and password file <none>
I20260928 15:57:16.200894 26092 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:16.201022 26092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:16.201328 26092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:16.204288 26092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "2069e49b8dd54a6eb5e4203bc4aa1407"
format_stamp: "Formatted at 2026-09-28 15:57:16 on dist-test-slave-9cqd"
server_key: "6231083d930a6a59ccb6519cbe860ec6"
server_key_iv: "cc66758943d927221023839fd2aa3ce4"
server_key_version: "encryptionkey@0"
I20260928 15:57:16.205024 26092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "2069e49b8dd54a6eb5e4203bc4aa1407"
format_stamp: "Formatted at 2026-09-28 15:57:16 on dist-test-slave-9cqd"
server_key: "6231083d930a6a59ccb6519cbe860ec6"
server_key_iv: "cc66758943d927221023839fd2aa3ce4"
server_key_version: "encryptionkey@0"
I20260928 15:57:16.209987 26092 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:16.213201 26107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:16.214464 26092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:16.214634 26092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "2069e49b8dd54a6eb5e4203bc4aa1407"
format_stamp: "Formatted at 2026-09-28 15:57:16 on dist-test-slave-9cqd"
server_key: "6231083d930a6a59ccb6519cbe860ec6"
server_key_iv: "cc66758943d927221023839fd2aa3ce4"
server_key_version: "encryptionkey@0"
I20260928 15:57:16.214821 26092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:16.234592 26092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:16.238512 26092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:16.238781 26092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:16.256256 26092 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:36341
I20260928 15:57:16.256287 26158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:36341 every 8 connection(s)
I20260928 15:57:16.257537 26092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:16.261036 26159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:16.266265 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26092
I20260928 15:57:16.266494 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:16.267259 23031 external_mini_cluster.cc:1428] Setting key 481b2217b9204073e69c7bb694ac24ec
I20260928 15:57:16.272153 26159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407: Bootstrap starting.
I20260928 15:57:16.275960 26159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:16.277235 26159 log.cc:824] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:16.280604 26159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407: No bootstrap required, opened a new log
Sep 28 15:57:16 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:16.291265 26159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36341 } }
I20260928 15:57:16.291728 26159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:16.291877 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:16.269322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.22.125.254:36341)
0928 15:57:16.270132 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:16.270142 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:16.270186 (+ 44us) server_negotiation.cc:407] Connection header received
0928 15:57:16.271725 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:16.271749 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:16.272155 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:16.272500 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:16.274418 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:16.275668 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:16.277618 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:16.278683 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:16.278808 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:16.283332 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:16.283344 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:16.283367 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:16.283410 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:16.287449 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:16.288095 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:16.288104 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:16.288114 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:16.288320 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:16.288667 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:16.288675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:16.288682 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:16.289373 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:16.289713 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:16.289992 (+ 279us) server_negotiation.cc:299] Negotiation successful
0928 15:57:16.290421 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":399,"threads_started":1}
I20260928 15:57:16.291851 26159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2069e49b8dd54a6eb5e4203bc4aa1407, State: Initialized, Role: FOLLOWER
I20260928 15:57:16.292521 26159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36341 } }
I20260928 15:57:16.292786 26159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:16.292939 26159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:16.293128 26159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:16.295869 26159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36341 } }
I20260928 15:57:16.296370 26159 leader_election.cc:302] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2069e49b8dd54a6eb5e4203bc4aa1407; no voters:
I20260928 15:57:16.296872 26159 leader_election.cc:288] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:16.297019 26164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:16.297717 26164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [term 1 LEADER]: Becoming Leader. State: Replica: 2069e49b8dd54a6eb5e4203bc4aa1407, State: Running, Role: LEADER
I20260928 15:57:16.298276 26164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36341 } }
I20260928 15:57:16.298913 26159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:16.301182 26165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36341 } } }
I20260928 15:57:16.301616 26165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:16.302402 26166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2069e49b8dd54a6eb5e4203bc4aa1407. Latest consensus state: current_term: 1 leader_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2069e49b8dd54a6eb5e4203bc4aa1407" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 36341 } } }
I20260928 15:57:16.302886 26166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:16.303076 26170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:16.308178 26170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:16.317225 26170 catalog_manager.cc:1347] Generated new cluster ID: eff0fff6e0aa43e280adc026cef07e87
I20260928 15:57:16.317328 26170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:16.353153 26170 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:16.354561 26170 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:16.370044 26170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2069e49b8dd54a6eb5e4203bc4aa1407: Generated new TSK 0
I20260928 15:57:16.370896 26170 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:57:16.426004 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36341
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:16.755409 26185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:16.755649 26185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:16.755748 26185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:16.762670 26185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:16.762766 26185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:16.762871 26185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:16.772102 26185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:16.773169 26185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:16.774521 26185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:16.787160 26191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:16.787125 26190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:16.787216 26193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:16.788002 26185 server_base.cc:1034] running on GCE node
I20260928 15:57:16.788843 26185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:16.789776 26185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:16.791006 26185 hybrid_clock.cc:648] HybridClock initialized: now 1790611036790942 us; error 39 us; skew 500 ppm
I20260928 15:57:16.791337 26185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:16 dist-test-slave-9cqd krb5kdc[26079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611036, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:16.794569 26185 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:57:16.795980 26185 webserver.cc:460] Webserver started at http://127.22.125.193:43201/ using document root <none> and password file <none>
I20260928 15:57:16.796648 26185 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:16.796794 26185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:16.797089 26185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:16.800117 26185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "b9c2a055cc0c487587f5e28b8bbece00"
format_stamp: "Formatted at 2026-09-28 15:57:16 on dist-test-slave-9cqd"
server_key: "46795a667592667bd5738aace8a39509"
server_key_iv: "fff72b998fd8e259722746ba26fddd04"
server_key_version: "encryptionkey@0"
I20260928 15:57:16.800958 26185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "b9c2a055cc0c487587f5e28b8bbece00"
format_stamp: "Formatted at 2026-09-28 15:57:16 on dist-test-slave-9cqd"
server_key: "46795a667592667bd5738aace8a39509"
server_key_iv: "fff72b998fd8e259722746ba26fddd04"
server_key_version: "encryptionkey@0"
I20260928 15:57:16.806160 26185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:16.809554 26200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:16.810859 26185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:57:16.811039 26185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "b9c2a055cc0c487587f5e28b8bbece00"
format_stamp: "Formatted at 2026-09-28 15:57:16 on dist-test-slave-9cqd"
server_key: "46795a667592667bd5738aace8a39509"
server_key_iv: "fff72b998fd8e259722746ba26fddd04"
server_key_version: "encryptionkey@0"
I20260928 15:57:16.811232 26185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:16.827921 26185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:16.832259 26185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:16.832546 26185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:16.833487 26185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:16.835039 26185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:16.835171 26185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:16.835310 26185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:16.835371 26185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:16.870791 26185 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:44825
I20260928 15:57:16.870884 26312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:44825 every 8 connection(s)
I20260928 15:57:16.872114 26185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:16.875586 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26185
I20260928 15:57:16.875838 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:16.876677 23031 external_mini_cluster.cc:1428] Setting key 6c53704c5fb84c51ff59a086c289bf23
Sep 28 15:57:16 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611036, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:16.894117 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:16.874653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:59813 (local address 127.22.125.254:36341)
0928 15:57:16.875114 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:16.875120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:16.876383 (+ 1263us) server_negotiation.cc:407] Connection header received
0928 15:57:16.878687 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:16.878693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:16.878777 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:16.878903 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:16.880533 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:16.881262 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:16.882865 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:16.883287 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:16.883434 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:16.887246 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:16.887256 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:16.887264 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:16.887311 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:16.890991 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:16.891704 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:16.891710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:16.891714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:16.891845 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:16.892340 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:16.892345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:16.892348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:16.892625 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:16.892782 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:16.893522 (+ 740us) server_negotiation.cc:299] Negotiation successful
0928 15:57:16.893761 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":224,"threads_started":1}
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:57:16.895174 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:16.875246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36341 (local address 127.22.125.193:59813)
0928 15:57:16.876191 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:16.876253 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:16.878386 (+ 2133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:16.879163 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:16.879172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:16.879698 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:16.880361 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:16.880381 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:16.881464 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:16.881468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:16.882713 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:16.882725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:16.883541 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:16.883548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:16.883777 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:16.884491 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:16.884514 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:16.886974 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:16.891144 (+ 4170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:16.891151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:16.891163 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:16.891566 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:16.891961 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:16.891967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:16.891971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:16.892219 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:16.892755 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:16.892763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:16.893133 (+ 370us) client_negotiation.cc:769] Sending connection context
0928 15:57:16.893402 (+ 269us) client_negotiation.cc:240] Negotiation successful
0928 15:57:16.893948 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":350,"threads_started":1}
I20260928 15:57:16.896893 26313 heartbeater.cc:347] Connected to a master server at 127.22.125.254:36341
I20260928 15:57:16.897229 26313 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:57:16.898044 26313 heartbeater.cc:511] Master 127.22.125.254:36341 requested a full tablet report, sending...
I20260928 15:57:16.900324 26124 ts_manager.cc:194] Registered new tserver with Master: b9c2a055cc0c487587f5e28b8bbece00 (127.22.125.193:44825)
I20260928 15:57:16.901762 26124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:59813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:57:16.920946 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36341
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:17.236414 26319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:17.236670 26319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:17.236760 26319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:17.243664 26319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:17.243765 26319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:17.243870 26319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:17.253108 26319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:17.254263 26319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:17.255441 26319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:17.268949 26324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:17.268980 26325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:17.269008 26327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:17.270324 26319 server_base.cc:1034] running on GCE node
I20260928 15:57:17.270998 26319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:17.272061 26319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:17.273319 26319 hybrid_clock.cc:648] HybridClock initialized: now 1790611037273247 us; error 44 us; skew 500 ppm
I20260928 15:57:17.273659 26319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:17.276839 26319 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:57:17.278131 26319 webserver.cc:460] Webserver started at http://127.22.125.194:46387/ using document root <none> and password file <none>
I20260928 15:57:17.278800 26319 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:17.278898 26319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:17.279136 26319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:17.281914 26319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "1fcd8fc24cb34fc9ae08eadf89f5a376"
format_stamp: "Formatted at 2026-09-28 15:57:17 on dist-test-slave-9cqd"
server_key: "6278bfce62fbb9cd6503fc5e4ac8f122"
server_key_iv: "240e0cef4fa271cbfa7bd80521a1c42d"
server_key_version: "encryptionkey@0"
I20260928 15:57:17.282632 26319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "1fcd8fc24cb34fc9ae08eadf89f5a376"
format_stamp: "Formatted at 2026-09-28 15:57:17 on dist-test-slave-9cqd"
server_key: "6278bfce62fbb9cd6503fc5e4ac8f122"
server_key_iv: "240e0cef4fa271cbfa7bd80521a1c42d"
server_key_version: "encryptionkey@0"
I20260928 15:57:17.287806 26319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 15:57:17.291141 26334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:17.292517 26319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:17.292650 26319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "1fcd8fc24cb34fc9ae08eadf89f5a376"
format_stamp: "Formatted at 2026-09-28 15:57:17 on dist-test-slave-9cqd"
server_key: "6278bfce62fbb9cd6503fc5e4ac8f122"
server_key_iv: "240e0cef4fa271cbfa7bd80521a1c42d"
server_key_version: "encryptionkey@0"
I20260928 15:57:17.292804 26319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:17.318389 26319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:17.322896 26319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:17.323200 26319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:17.324241 26319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:17.325569 26319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:17.325675 26319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:17.325835 26319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:17.325917 26319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:17.358753 26319 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:38351
I20260928 15:57:17.358801 26446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:38351 every 8 connection(s)
I20260928 15:57:17.360116 26319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:17.368450 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26319
I20260928 15:57:17.368659 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:17.369232 23031 external_mini_cluster.cc:1428] Setting key 485295e448d193e74f29d67460e2db08
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:17.381304 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:17.362738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:39177 (local address 127.22.125.254:36341)
0928 15:57:17.362943 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:17.362949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:17.364128 (+ 1179us) server_negotiation.cc:407] Connection header received
0928 15:57:17.365856 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:17.365860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:17.365931 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:17.366022 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:17.367534 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.368340 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.370479 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.371176 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.371316 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:17.375378 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:17.375383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:17.375386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:17.375419 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:17.378149 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.378902 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.378908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.378913 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.379045 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.379478 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.379483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.379486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.379758 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:17.379946 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:17.380727 (+ 781us) server_negotiation.cc:299] Negotiation successful
0928 15:57:17.380960 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 15:57:17.382328 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:17.363244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36341 (local address 127.22.125.194:39177)
0928 15:57:17.363946 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:17.363984 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:17.365617 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:17.366235 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:17.366244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:17.366722 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:17.367381 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:17.367399 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.368508 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.368516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:17.370307 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:17.370322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.371417 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.371425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:17.371694 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:17.372435 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:17.372470 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:17.375150 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:17.378309 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:17.378318 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:17.378335 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:17.378768 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:17.379160 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:17.379165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:17.379169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:17.379360 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:17.379925 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:17.379933 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:17.380368 (+ 435us) client_negotiation.cc:769] Sending connection context
0928 15:57:17.380642 (+ 274us) client_negotiation.cc:240] Negotiation successful
0928 15:57:17.381201 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":280,"threads_started":1}
I20260928 15:57:17.383962 26447 heartbeater.cc:347] Connected to a master server at 127.22.125.254:36341
I20260928 15:57:17.384306 26447 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:17.385037 26447 heartbeater.cc:511] Master 127.22.125.254:36341 requested a full tablet report, sending...
I20260928 15:57:17.386667 26124 ts_manager.cc:194] Registered new tserver with Master: 1fcd8fc24cb34fc9ae08eadf89f5a376 (127.22.125.194:38351)
I20260928 15:57:17.387939 26124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:39177
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:57:17.413339 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36341
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:17.729990 26452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:17.730316 26452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:17.730494 26452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:17.737685 26452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:17.737835 26452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:17.738044 26452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:17.747529 26452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:36341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:17.748898 26452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:17.750248 26452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:17.763082 26457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:17.763072 26460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:17.763236 26458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:17.764178 26452 server_base.cc:1034] running on GCE node
I20260928 15:57:17.764757 26452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:17.765615 26452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:17.766839 26452 hybrid_clock.cc:648] HybridClock initialized: now 1790611037766800 us; error 53 us; skew 500 ppm
I20260928 15:57:17.767139 26452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:17.770144 26452 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:57:17.771559 26452 webserver.cc:460] Webserver started at http://127.22.125.195:40967/ using document root <none> and password file <none>
I20260928 15:57:17.772302 26452 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:17.772456 26452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:17.772753 26452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:17.775560 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "00e63c7d3eed434e8896ca4aec2eb168"
format_stamp: "Formatted at 2026-09-28 15:57:17 on dist-test-slave-9cqd"
server_key: "01a1eb6e7552eca2d4818512131463af"
server_key_iv: "8f37a0b086079de39e79602a65956476"
server_key_version: "encryptionkey@0"
I20260928 15:57:17.776417 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "00e63c7d3eed434e8896ca4aec2eb168"
format_stamp: "Formatted at 2026-09-28 15:57:17 on dist-test-slave-9cqd"
server_key: "01a1eb6e7552eca2d4818512131463af"
server_key_iv: "8f37a0b086079de39e79602a65956476"
server_key_version: "encryptionkey@0"
I20260928 15:57:17.781306 26452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 15:57:17.784894 26467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:17.786161 26452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:57:17.786334 26452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "00e63c7d3eed434e8896ca4aec2eb168"
format_stamp: "Formatted at 2026-09-28 15:57:17 on dist-test-slave-9cqd"
server_key: "01a1eb6e7552eca2d4818512131463af"
server_key_iv: "8f37a0b086079de39e79602a65956476"
server_key_version: "encryptionkey@0"
I20260928 15:57:17.786526 26452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:17.812026 26452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:17.815903 26452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:17.816226 26452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:17.817148 26452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:17.818447 26452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:17.818547 26452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:17.818691 26452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:17.818763 26452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:17.851483 26452 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:33171
I20260928 15:57:17.851523 26579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:33171 every 8 connection(s)
I20260928 15:57:17.852962 26452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:17.860136 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26452
I20260928 15:57:17.860348 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:17.860918 23031 external_mini_cluster.cc:1428] Setting key 2b8bc1445f78c688feabaf38393e4985
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:17.872998 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:17.855584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:59985 (local address 127.22.125.254:36341)
0928 15:57:17.855762 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:17.855767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:17.857002 (+ 1235us) server_negotiation.cc:407] Connection header received
0928 15:57:17.858758 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:17.858762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:17.858844 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:17.858940 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:17.860730 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.861555 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.863787 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.864257 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.864358 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:17.867589 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:17.867593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:17.867596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:17.867624 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:17.870088 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.870878 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.870881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.870883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.871002 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.871494 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.871497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.871499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.871729 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:17.871831 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:17.872614 (+ 783us) server_negotiation.cc:299] Negotiation successful
0928 15:57:17.872764 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 15:57:17.874326 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:17.856156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:36341 (local address 127.22.125.195:59985)
0928 15:57:17.856842 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:17.856881 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:17.858489 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:17.859131 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:17.859138 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:17.859658 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:17.860547 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:17.860575 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.861734 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.861746 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:17.863651 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:17.863673 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.864447 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.864452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:17.864639 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:17.865308 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:17.865329 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:17.867422 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:17.870237 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:17.870245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:17.870259 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:17.870764 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:17.871112 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:17.871119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:17.871124 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:17.871379 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:17.871844 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:17.871851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:17.872250 (+ 399us) client_negotiation.cc:769] Sending connection context
0928 15:57:17.872545 (+ 295us) client_negotiation.cc:240] Negotiation successful
0928 15:57:17.873025 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":264,"threads_started":1}
I20260928 15:57:17.875948 26580 heartbeater.cc:347] Connected to a master server at 127.22.125.254:36341
I20260928 15:57:17.876262 26580 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:17.876964 26580 heartbeater.cc:511] Master 127.22.125.254:36341 requested a full tablet report, sending...
I20260928 15:57:17.878347 26124 ts_manager.cc:194] Registered new tserver with Master: 00e63c7d3eed434e8896ca4aec2eb168 (127.22.125.195:33171)
I20260928 15:57:17.879144 26124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:59985
I20260928 15:57:17.888108 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:57:17.904603 26313 heartbeater.cc:503] Master 127.22.125.254:36341 was elected leader, sending a full tablet report...
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:17.933418 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:17.920312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.22.125.254:36341)
0928 15:57:17.920490 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:17.920494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:17.921094 (+ 600us) server_negotiation.cc:407] Connection header received
0928 15:57:17.921244 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:17.921248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:17.921311 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:17.921399 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:17.922742 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.923451 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.924733 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.925120 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.925213 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:17.928906 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:17.928911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:17.928914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:17.928944 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:17.931257 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.931795 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.931799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.931801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.931934 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.932321 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.932324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.932326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.932617 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:17.932798 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:17.933044 (+ 246us) server_negotiation.cc:299] Negotiation successful
0928 15:57:17.933196 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260928 15:57:17.933960 26124 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43174
I20260928 15:57:17.935541 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26185
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:17.950482 26591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:17.936629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.22.125.193:44825)
0928 15:57:17.937267 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:17.937284 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:17.937307 (+ 23us) server_negotiation.cc:407] Connection header received
0928 15:57:17.937360 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:17.937367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:17.937567 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:17.937729 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:17.938884 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.939894 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.941083 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.941637 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.941784 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:17.945365 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:17.945376 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:17.945390 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:17.945420 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:17.948324 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.948879 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.948885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.948888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.949001 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.949296 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.949299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.949301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.949608 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:17.949825 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:17.950025 (+ 200us) server_negotiation.cc:299] Negotiation successful
0928 15:57:17.950205 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":298,"threads_started":1}
W20260928 15:57:17.951710 26308 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60748
W20260928 15:57:17.952929 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26185: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:57:17.953118 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26185
I20260928 15:57:17.969310 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26319
Sep 28 15:57:17 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:57:17.983824 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:17.970108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37948 (local address 127.22.125.194:38351)
0928 15:57:17.970618 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:17.970624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:17.970649 (+ 25us) server_negotiation.cc:407] Connection header received
0928 15:57:17.970705 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:17.970710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:17.970915 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:17.971046 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:17.972183 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.973137 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.974332 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:17.974842 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:17.974947 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:17.978700 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:17.978711 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:17.978723 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:17.978762 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:17.981578 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.982132 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.982138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.982140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.982267 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:17.982572 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:17.982575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:17.982577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:17.982914 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:17.983131 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:17.983346 (+ 215us) server_negotiation.cc:299] Negotiation successful
0928 15:57:17.983540 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":268,"threads_started":1}
W20260928 15:57:17.985163 26442 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37948
W20260928 15:57:17.986196 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26319: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:57:17.986388 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26319
I20260928 15:57:18.001658 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26452
Sep 28 15:57:18 dist-test-slave-9cqd krb5kdc[26079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611037, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:57:18.016999 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:18.002498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.22.125.195:33171)
0928 15:57:18.003153 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:18.003160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:18.003188 (+ 28us) server_negotiation.cc:407] Connection header received
0928 15:57:18.003245 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:18.003251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:18.003457 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:18.003579 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:18.004813 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:18.005836 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:18.007014 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:18.007681 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:18.007774 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:18.011925 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:18.011936 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:18.011950 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:18.011986 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:18.014596 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:18.015185 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:18.015191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:18.015193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:18.015308 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:18.015601 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:18.015604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:18.015606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:18.015929 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:18.016229 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:18.016404 (+ 175us) server_negotiation.cc:299] Negotiation successful
0928 15:57:18.016595 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":324,"threads_started":1}
W20260928 15:57:18.018277 26575 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51634
W20260928 15:57:18.019443 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26452: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 15:57:18.019600 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26452
I20260928 15:57:18.035395 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26092
I20260928 15:57:18.036631 26154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:18.141053 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26092
2026-09-28T15:57:18Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3580 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:18 dist-test-slave-9cqd krb5kdc[26597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:18 dist-test-slave-9cqd krb5kdc[26597](info): set up 2 sockets
Sep 28 15:57:18 dist-test-slave-9cqd krb5kdc[26597](info): commencing operation
krb5kdc: starting...
W20260928 15:57:19.270344 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:19 dist-test-slave-9cqd krb5kdc[26597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:19.460290 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44081
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:38613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:44081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:19.782518 26610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:19.782783 26610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:19.782975 26610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:19.790076 26610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:19.790174 26610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:19.790220 26610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:19.790302 26610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:19.790390 26610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:19.799755 26610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:38613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:44081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:19.801069 26610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:19.802378 26610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:19.814188 26616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:19.814208 26618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:19.814306 26615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:19.815416 26610 server_base.cc:1034] running on GCE node
I20260928 15:57:19.816274 26610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:19.817629 26610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:19.818866 26610 hybrid_clock.cc:648] HybridClock initialized: now 1790611039818818 us; error 57 us; skew 500 ppm
I20260928 15:57:19.819214 26610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:19 dist-test-slave-9cqd krb5kdc[26597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611039, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:19.822254 26610 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:19.823916 26610 webserver.cc:460] Webserver started at http://127.22.125.254:35003/ using document root <none> and password file <none>
I20260928 15:57:19.824653 26610 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:19.824801 26610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:19.825111 26610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:19.828516 26610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "ac860e3da92b4fe59387345cb7efea56"
format_stamp: "Formatted at 2026-09-28 15:57:19 on dist-test-slave-9cqd"
server_key: "76cfd13506a5ac6394cadc0d1baa8ead"
server_key_iv: "1d9cb87f3c7bfb3dd5cd5e285e6f04f2"
server_key_version: "encryptionkey@0"
I20260928 15:57:19.829399 26610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "ac860e3da92b4fe59387345cb7efea56"
format_stamp: "Formatted at 2026-09-28 15:57:19 on dist-test-slave-9cqd"
server_key: "76cfd13506a5ac6394cadc0d1baa8ead"
server_key_iv: "1d9cb87f3c7bfb3dd5cd5e285e6f04f2"
server_key_version: "encryptionkey@0"
I20260928 15:57:19.834487 26610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:19.837841 26625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:19.839212 26610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260928 15:57:19.839460 26610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "ac860e3da92b4fe59387345cb7efea56"
format_stamp: "Formatted at 2026-09-28 15:57:19 on dist-test-slave-9cqd"
server_key: "76cfd13506a5ac6394cadc0d1baa8ead"
server_key_iv: "1d9cb87f3c7bfb3dd5cd5e285e6f04f2"
server_key_version: "encryptionkey@0"
I20260928 15:57:19.839682 26610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:19.866375 26610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:19.870613 26610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:19.870934 26610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:19.889288 26610 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:44081
I20260928 15:57:19.889385 26676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:44081 every 8 connection(s)
I20260928 15:57:19.890683 26610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:19.894356 26677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:19.897986 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26610
I20260928 15:57:19.898200 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:19.898867 23031 external_mini_cluster.cc:1428] Setting key 5ce5fb1f2c8f8649bee0f6273180a487
I20260928 15:57:19.906575 26677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56: Bootstrap starting.
I20260928 15:57:19.910202 26677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:19.911464 26677 log.cc:824] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:19.914299 26677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56: No bootstrap required, opened a new log
Sep 28 15:57:19 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:19.924196 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:19.900918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.22.125.254:44081)
0928 15:57:19.901720 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:19.901735 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:19.901786 (+ 51us) server_negotiation.cc:407] Connection header received
0928 15:57:19.904775 (+ 2989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:19.904802 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:19.905313 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:19.905743 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:19.907216 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:19.909294 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:19.910558 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:19.911620 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:19.911848 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:19.915687 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:19.915699 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:19.915738 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:19.915793 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:19.919756 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:19.920446 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:19.920454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:19.920462 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:19.920690 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:19.921050 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:19.921055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:19.921059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:19.921774 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:19.922071 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:19.922366 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 15:57:19.922820 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":338,"threads_started":1}
I20260928 15:57:19.925086 26677 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac860e3da92b4fe59387345cb7efea56" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44081 } }
I20260928 15:57:19.925438 26677 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:19.925566 26677 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac860e3da92b4fe59387345cb7efea56, State: Initialized, Role: FOLLOWER
I20260928 15:57:19.926194 26677 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac860e3da92b4fe59387345cb7efea56" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44081 } }
I20260928 15:57:19.926440 26677 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:19.926594 26677 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:19.926795 26677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:19.928942 26677 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac860e3da92b4fe59387345cb7efea56" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44081 } }
I20260928 15:57:19.929407 26677 leader_election.cc:302] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac860e3da92b4fe59387345cb7efea56; no voters:
I20260928 15:57:19.929952 26677 leader_election.cc:288] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:19.930080 26682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:19.930781 26682 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [term 1 LEADER]: Becoming Leader. State: Replica: ac860e3da92b4fe59387345cb7efea56, State: Running, Role: LEADER
I20260928 15:57:19.931280 26682 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac860e3da92b4fe59387345cb7efea56" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44081 } }
I20260928 15:57:19.931977 26677 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:19.933903 26684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac860e3da92b4fe59387345cb7efea56. Latest consensus state: current_term: 1 leader_uuid: "ac860e3da92b4fe59387345cb7efea56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac860e3da92b4fe59387345cb7efea56" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44081 } } }
I20260928 15:57:19.934340 26684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:19.934247 26683 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac860e3da92b4fe59387345cb7efea56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac860e3da92b4fe59387345cb7efea56" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44081 } } }
I20260928 15:57:19.934634 26683 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:19.936707 26689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:19.941156 26689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:19.949589 26689 catalog_manager.cc:1347] Generated new cluster ID: f9474c25950c49edbb345d85bff2c1c9
I20260928 15:57:19.949685 26689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:19.971488 26689 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:19.972878 26689 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:19.978577 26689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac860e3da92b4fe59387345cb7efea56: Generated new TSK 0
I20260928 15:57:19.979544 26689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:57:20.022970 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44081
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:38613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:20.346164 26703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:20.346410 26703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:20.346501 26703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:20.353643 26703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:20.353739 26703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:20.353845 26703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:20.363165 26703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:38613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:20.364396 26703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:20.365651 26703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:20.378659 26711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:20.378563 26709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:20.378741 26708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:20.379726 26703 server_base.cc:1034] running on GCE node
I20260928 15:57:20.380378 26703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:20.381389 26703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:20.382616 26703 hybrid_clock.cc:648] HybridClock initialized: now 1790611040382600 us; error 132 us; skew 500 ppm
I20260928 15:57:20.382927 26703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:20 dist-test-slave-9cqd krb5kdc[26597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611040, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:20.386065 26703 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:57:20.387799 26703 webserver.cc:460] Webserver started at http://127.22.125.193:45449/ using document root <none> and password file <none>
I20260928 15:57:20.388530 26703 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:20.388679 26703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:20.388989 26703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:20.391764 26703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "e692929f146346c49c9aade6a1fff107"
format_stamp: "Formatted at 2026-09-28 15:57:20 on dist-test-slave-9cqd"
server_key: "33d879064d17967a0d740481185e6a65"
server_key_iv: "eda68830bf645a535a3f7aff31be0613"
server_key_version: "encryptionkey@0"
I20260928 15:57:20.392592 26703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "e692929f146346c49c9aade6a1fff107"
format_stamp: "Formatted at 2026-09-28 15:57:20 on dist-test-slave-9cqd"
server_key: "33d879064d17967a0d740481185e6a65"
server_key_iv: "eda68830bf645a535a3f7aff31be0613"
server_key_version: "encryptionkey@0"
I20260928 15:57:20.397433 26703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260928 15:57:20.400698 26718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:20.401929 26703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:20.402097 26703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "e692929f146346c49c9aade6a1fff107"
format_stamp: "Formatted at 2026-09-28 15:57:20 on dist-test-slave-9cqd"
server_key: "33d879064d17967a0d740481185e6a65"
server_key_iv: "eda68830bf645a535a3f7aff31be0613"
server_key_version: "encryptionkey@0"
I20260928 15:57:20.402305 26703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:20.421546 26703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:20.426139 26703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:20.426436 26703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:20.427336 26703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:20.428699 26703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:20.428803 26703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:20.428954 26703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:20.429028 26703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:20.461899 26703 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:38167
I20260928 15:57:20.461938 26830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:38167 every 8 connection(s)
I20260928 15:57:20.463147 26703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:20.466101 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26703
I20260928 15:57:20.466472 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:20.467164 23031 external_mini_cluster.cc:1428] Setting key 19f2532c673dbc50275e2eab3274404f
Sep 28 15:57:20 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611040, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:20.485409 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:20.465825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:55681 (local address 127.22.125.254:44081)
0928 15:57:20.466311 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:20.466319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:20.467458 (+ 1139us) server_negotiation.cc:407] Connection header received
0928 15:57:20.469603 (+ 2145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:20.469609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:20.469701 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:20.469820 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:20.471248 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.472418 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.473990 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.474607 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.474712 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:20.479039 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:20.479043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:20.479046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:20.479076 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:20.481994 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:20.482739 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:20.482746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:20.482750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:20.482891 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:20.483317 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:20.483323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:20.483327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:20.483594 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:20.483752 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:20.484805 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0928 15:57:20.485049 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":231,"threads_started":1}
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:57:20.486461 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:20.466422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44081 (local address 127.22.125.193:55681)
0928 15:57:20.467283 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:20.467335 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:20.469350 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:20.470033 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:20.470041 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:20.470528 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:20.471076 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:20.471092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.472556 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.472560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:20.473835 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:20.473847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.474840 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.474848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:20.475168 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:20.475914 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:20.475953 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:20.478793 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:20.482145 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:20.482152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:20.482163 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:20.482601 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:20.483005 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:20.483010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:20.483014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:20.483198 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:20.483728 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:20.483736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:20.484106 (+ 370us) client_negotiation.cc:769] Sending connection context
0928 15:57:20.484731 (+ 625us) client_negotiation.cc:240] Negotiation successful
0928 15:57:20.485291 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":395,"threads_started":1}
I20260928 15:57:20.487944 26831 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44081
I20260928 15:57:20.488286 26831 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:20.489056 26831 heartbeater.cc:511] Master 127.22.125.254:44081 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:57:20.491122 26642 ts_manager.cc:194] Registered new tserver with Master: e692929f146346c49c9aade6a1fff107 (127.22.125.193:38167)
I20260928 15:57:20.492628 26642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:55681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:57:20.511307 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44081
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:38613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:20.829016 26837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:20.829286 26837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:20.829380 26837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:20.836612 26837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:20.836717 26837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:20.836880 26837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:20.846405 26837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:38613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:20.847687 26837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:20.848927 26837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:20.862052 26842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:20.861992 26843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:20.862051 26845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:20.862762 26837 server_base.cc:1034] running on GCE node
I20260928 15:57:20.863356 26837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:20.864331 26837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:20.865561 26837 hybrid_clock.cc:648] HybridClock initialized: now 1790611040865520 us; error 38 us; skew 500 ppm
I20260928 15:57:20.865875 26837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:20 dist-test-slave-9cqd krb5kdc[26597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611040, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:20.868981 26837 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:57:20.870512 26837 webserver.cc:460] Webserver started at http://127.22.125.194:46785/ using document root <none> and password file <none>
I20260928 15:57:20.871168 26837 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:20.871307 26837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:20.871604 26837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:20.874480 26837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "754c73ed9833431c856c9370db9f2837"
format_stamp: "Formatted at 2026-09-28 15:57:20 on dist-test-slave-9cqd"
server_key: "20f7fc68dd2d620d56cfcef06c16ff7f"
server_key_iv: "e53e0fbabbb9f59f20e405c55287f4d9"
server_key_version: "encryptionkey@0"
I20260928 15:57:20.875393 26837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "754c73ed9833431c856c9370db9f2837"
format_stamp: "Formatted at 2026-09-28 15:57:20 on dist-test-slave-9cqd"
server_key: "20f7fc68dd2d620d56cfcef06c16ff7f"
server_key_iv: "e53e0fbabbb9f59f20e405c55287f4d9"
server_key_version: "encryptionkey@0"
I20260928 15:57:20.880326 26837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260928 15:57:20.883529 26852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:20.884889 26837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:20.885070 26837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "754c73ed9833431c856c9370db9f2837"
format_stamp: "Formatted at 2026-09-28 15:57:20 on dist-test-slave-9cqd"
server_key: "20f7fc68dd2d620d56cfcef06c16ff7f"
server_key_iv: "e53e0fbabbb9f59f20e405c55287f4d9"
server_key_version: "encryptionkey@0"
I20260928 15:57:20.885272 26837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:20.912322 26837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:20.916177 26837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:20.916476 26837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:20.917408 26837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:20.918670 26837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:20.918777 26837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:20.918926 26837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:20.918998 26837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:20.951610 26837 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:33849
I20260928 15:57:20.951697 26964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:33849 every 8 connection(s)
I20260928 15:57:20.953042 26837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:20.958654 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26837
I20260928 15:57:20.958827 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:20.959359 23031 external_mini_cluster.cc:1428] Setting key 0addd642f70748277ce5e4da463cd555
Sep 28 15:57:20 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611040, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:20.973934 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:20.955556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:60403 (local address 127.22.125.254:44081)
0928 15:57:20.955722 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:20.955727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:20.956956 (+ 1229us) server_negotiation.cc:407] Connection header received
0928 15:57:20.958906 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:20.958914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:20.959020 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:20.959108 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:20.960995 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.961736 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.963247 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.963632 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.963759 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:20.967167 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:20.967176 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:20.967183 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:20.967227 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:20.970850 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:20.971541 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:20.971549 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:20.971556 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:20.971725 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:20.972158 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:20.972165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:20.972171 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:20.972538 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:20.972696 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:20.973353 (+ 657us) server_negotiation.cc:299] Negotiation successful
0928 15:57:20.973593 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 15:57:20.974995 26967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:20.956110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44081 (local address 127.22.125.194:60403)
0928 15:57:20.956785 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:20.956827 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:20.958559 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:20.959392 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:20.959404 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:20.960141 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:20.960827 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:20.960845 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.961881 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.961886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:20.963092 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:20.963103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:20.963858 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:20.963867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:20.964143 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:20.964817 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:20.964838 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:20.966966 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:20.970979 (+ 4013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:20.970985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:20.970997 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:20.971404 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:20.971829 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:20.971832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:20.971834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:20.972013 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:20.972664 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:20.972670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:20.972984 (+ 314us) client_negotiation.cc:769] Sending connection context
0928 15:57:20.973245 (+ 261us) client_negotiation.cc:240] Negotiation successful
0928 15:57:20.973760 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":261,"threads_started":1}
I20260928 15:57:20.976442 26965 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44081
I20260928 15:57:20.976783 26965 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:20.977680 26965 heartbeater.cc:511] Master 127.22.125.254:44081 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 15:57:20.979341 26642 ts_manager.cc:194] Registered new tserver with Master: 754c73ed9833431c856c9370db9f2837 (127.22.125.194:33849)
I20260928 15:57:20.980284 26642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:60403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:57:21.002628 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44081
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:38613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:21.318127 26970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:21.318439 26970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:21.318599 26970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:21.325531 26970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:21.325673 26970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:21.325855 26970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:21.335214 26970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:38613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:21.336467 26970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:21.337899 26970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:21.350703 26975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:21.350737 26976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:21.350802 26978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:21.351388 26970 server_base.cc:1034] running on GCE node
I20260928 15:57:21.352151 26970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:21.353098 26970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:21.354331 26970 hybrid_clock.cc:648] HybridClock initialized: now 1790611041354286 us; error 56 us; skew 500 ppm
I20260928 15:57:21.354650 26970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:21 dist-test-slave-9cqd krb5kdc[26597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611041, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:21.363979 26970 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:57:21.365437 26970 webserver.cc:460] Webserver started at http://127.22.125.195:44737/ using document root <none> and password file <none>
I20260928 15:57:21.366144 26970 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:21.366290 26970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:21.366611 26970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:21.369642 26970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "09bb55c6ae3742c180744152a63e283b"
format_stamp: "Formatted at 2026-09-28 15:57:21 on dist-test-slave-9cqd"
server_key: "7b32f4755f8342c1e12294dfe2eb6ae3"
server_key_iv: "4cedf9e5a3c909e81ab2ff0fc26194c9"
server_key_version: "encryptionkey@0"
I20260928 15:57:21.370421 26970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "09bb55c6ae3742c180744152a63e283b"
format_stamp: "Formatted at 2026-09-28 15:57:21 on dist-test-slave-9cqd"
server_key: "7b32f4755f8342c1e12294dfe2eb6ae3"
server_key_iv: "4cedf9e5a3c909e81ab2ff0fc26194c9"
server_key_version: "encryptionkey@0"
I20260928 15:57:21.375540 26970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 15:57:21.378998 26985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:21.380417 26970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:21.380590 26970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "09bb55c6ae3742c180744152a63e283b"
format_stamp: "Formatted at 2026-09-28 15:57:21 on dist-test-slave-9cqd"
server_key: "7b32f4755f8342c1e12294dfe2eb6ae3"
server_key_iv: "4cedf9e5a3c909e81ab2ff0fc26194c9"
server_key_version: "encryptionkey@0"
I20260928 15:57:21.380813 26970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:21.409888 26970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:21.413821 26970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:21.414124 26970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:21.415097 26970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:21.416455 26970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:21.416558 26970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:21.416704 26970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:21.416786 26970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:21.449136 26970 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:34463
I20260928 15:57:21.449185 27097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:34463 every 8 connection(s)
I20260928 15:57:21.450393 26970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:21.460801 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 26970
I20260928 15:57:21.460983 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:21.461648 23031 external_mini_cluster.cc:1428] Setting key 5118de5f75a968ebcb08bef5c8c140c9
Sep 28 15:57:21 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611041, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:21.470109 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:21.452894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:33645 (local address 127.22.125.254:44081)
0928 15:57:21.453096 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:21.453102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:21.454301 (+ 1199us) server_negotiation.cc:407] Connection header received
0928 15:57:21.456236 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:21.456241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:21.456307 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:21.456400 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:21.457818 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.458558 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.460001 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.460465 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.460558 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:21.464872 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:21.464879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:21.464884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:21.464929 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:21.467567 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.468267 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.468270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.468272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.468402 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.468766 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.468769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.468771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.469034 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:21.469153 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:21.469724 (+ 571us) server_negotiation.cc:299] Negotiation successful
0928 15:57:21.469879 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 15:57:21.470973 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:21.453480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44081 (local address 127.22.125.195:33645)
0928 15:57:21.454144 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:21.454189 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:21.455901 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:21.456605 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:21.456613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:21.457086 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:21.457669 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:21.457686 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.458681 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.458685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:21.459885 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:21.459897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.460691 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.460701 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:21.461031 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:21.461830 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:21.461869 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:21.464673 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:21.467698 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:21.467704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:21.467715 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:21.468168 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:21.468493 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:21.468496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:21.468498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:21.468667 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:21.469134 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:21.469140 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:21.469423 (+ 283us) client_negotiation.cc:769] Sending connection context
0928 15:57:21.469623 (+ 200us) client_negotiation.cc:240] Negotiation successful
0928 15:57:21.469960 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":274,"threads_started":1}
I20260928 15:57:21.472278 27098 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44081
I20260928 15:57:21.472563 27098 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:21.473222 27098 heartbeater.cc:511] Master 127.22.125.254:44081 requested a full tablet report, sending...
I20260928 15:57:21.474519 26642 ts_manager.cc:194] Registered new tserver with Master: 09bb55c6ae3742c180744152a63e283b (127.22.125.195:34463)
I20260928 15:57:21.475557 26642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:33645
I20260928 15:57:21.477084 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:57:21.495718 26831 heartbeater.cc:503] Master 127.22.125.254:44081 was elected leader, sending a full tablet report...
Sep 28 15:57:21 dist-test-slave-9cqd krb5kdc[26597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 15:57:21 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:21.524303 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:21.509784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.22.125.193:38167)
0928 15:57:21.510440 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:21.510447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:21.510473 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:57:21.510533 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:21.510539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:21.510764 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:21.510891 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:21.512349 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.513354 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.514561 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.515115 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.515209 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:21.518987 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:21.519001 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:21.519016 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:21.519049 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:21.521895 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.522482 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.522488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.522490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.522607 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.522908 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.522911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.522913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.523227 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:21.523458 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:21.523693 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 15:57:21.523883 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":324,"threads_started":1}
Sep 28 15:57:21 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:21.541651 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:21.528526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.22.125.254:44081)
0928 15:57:21.528743 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:21.528751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:21.528969 (+ 218us) server_negotiation.cc:407] Connection header received
0928 15:57:21.529156 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:21.529161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:21.529223 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:21.529325 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:21.530487 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.531207 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.532600 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.533082 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.533213 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:21.537074 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:21.537081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:21.537088 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:21.537125 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:21.539506 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.540101 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.540106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.540108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.540236 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.540551 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.540554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.540556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.540841 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:21.541011 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:21.541247 (+ 236us) server_negotiation.cc:299] Negotiation successful
0928 15:57:21.541428 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260928 15:57:21.542225 26642 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49262
I20260928 15:57:21.543289 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26703
I20260928 15:57:21.556110 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:21.544154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.22.125.193:38167)
0928 15:57:21.544372 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:21.544381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:21.544649 (+ 268us) server_negotiation.cc:407] Connection header received
0928 15:57:21.544854 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:21.544861 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:21.544955 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:21.545058 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:21.546263 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.546938 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.548182 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.548601 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.548742 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:21.550894 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:21.550899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:21.550902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:21.550932 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:21.553302 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.553922 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.553927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.553932 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.554052 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.554417 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.554423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.554426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.554662 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:21.554904 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:21.555211 (+ 307us) server_negotiation.cc:299] Negotiation successful
0928 15:57:21.555443 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 15:57:21.556684 26826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:21.639807 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26703
I20260928 15:57:21.657291 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26837
Sep 28 15:57:21 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:57:21.673202 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:21.658198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54590 (local address 127.22.125.194:33849)
0928 15:57:21.658798 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:21.658805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:21.658831 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:57:21.658900 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:21.658906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:21.659122 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:21.659252 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:21.660592 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.661563 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.662827 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.663364 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.663455 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:21.667728 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:21.667739 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:21.667752 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:21.667789 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:21.670600 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.671205 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.671212 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.671214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.671328 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.671811 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.671814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.671816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.672245 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:21.672472 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:21.672727 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 15:57:21.672925 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":320,"threads_started":1}
I20260928 15:57:21.674448 26960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:21.756860 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26837
I20260928 15:57:21.772553 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26970
Sep 28 15:57:21 dist-test-slave-9cqd krb5kdc[26597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:57:21.789134 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:21.773493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.22.125.195:34463)
0928 15:57:21.774128 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:21.774136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:21.774163 (+ 27us) server_negotiation.cc:407] Connection header received
0928 15:57:21.774222 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:21.774229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:21.774453 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:21.774634 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:21.775982 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.777054 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.778293 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:21.778851 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:21.778996 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:21.783410 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:21.783420 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:21.783433 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:21.783488 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:21.786326 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.786972 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.786978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.786980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.787096 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:21.787476 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:21.787486 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:21.787488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:21.787816 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:21.788149 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:21.788546 (+ 397us) server_negotiation.cc:299] Negotiation successful
0928 15:57:21.788805 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":338,"threads_started":1}
I20260928 15:57:21.790347 27093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:21.877840 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26970
I20260928 15:57:21.893713 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 26610
I20260928 15:57:21.894997 26672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:21.996933 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 26610
2026-09-28T15:57:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3856 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:22 dist-test-slave-9cqd krb5kdc[27117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:22 dist-test-slave-9cqd krb5kdc[27117](info): set up 2 sockets
Sep 28 15:57:22 dist-test-slave-9cqd krb5kdc[27117](info): commencing operation
krb5kdc: starting...
W20260928 15:57:23.094309 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:23 dist-test-slave-9cqd krb5kdc[27117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:23.277807 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42567
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:42567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:23.593370 27130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:23.593699 27130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:23.593876 27130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:23.600941 27130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:23.601081 27130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:23.601202 27130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:23.601306 27130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:23.601408 27130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:23.611409 27130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:42567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:23.612807 27130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:23.614208 27130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:23.625708 27136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:23.625801 27135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:23.625771 27138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:23.626514 27130 server_base.cc:1034] running on GCE node
I20260928 15:57:23.627265 27130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:23.628614 27130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:23.629807 27130 hybrid_clock.cc:648] HybridClock initialized: now 1790611043629752 us; error 46 us; skew 500 ppm
I20260928 15:57:23.630144 27130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:23 dist-test-slave-9cqd krb5kdc[27117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611043, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:23.633117 27130 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:23.634119 27130 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:23.634205 27130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:23.634471 27130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:23.637303 27130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb"
format_stamp: "Formatted at 2026-09-28 15:57:23 on dist-test-slave-9cqd"
server_key: "9ae2d9a2a2247fe8f5471257a700be41"
server_key_iv: "9947c435b32bcb982d2f80d484eebc69"
server_key_version: "encryptionkey@0"
I20260928 15:57:23.638087 27130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb"
format_stamp: "Formatted at 2026-09-28 15:57:23 on dist-test-slave-9cqd"
server_key: "9ae2d9a2a2247fe8f5471257a700be41"
server_key_iv: "9947c435b32bcb982d2f80d484eebc69"
server_key_version: "encryptionkey@0"
I20260928 15:57:23.642824 27130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260928 15:57:23.646010 27144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:23.647233 27130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:23.647414 27130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb"
format_stamp: "Formatted at 2026-09-28 15:57:23 on dist-test-slave-9cqd"
server_key: "9ae2d9a2a2247fe8f5471257a700be41"
server_key_iv: "9947c435b32bcb982d2f80d484eebc69"
server_key_version: "encryptionkey@0"
I20260928 15:57:23.647624 27130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:23.673630 27130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:23.677531 27130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:23.677805 27130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:23.695811 27130 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:42567
I20260928 15:57:23.695834 27195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:42567 every 8 connection(s)
I20260928 15:57:23.696745 27130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:23.700421 27196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:23.705310 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 27130
I20260928 15:57:23.705524 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:23.706180 23031 external_mini_cluster.cc:1428] Setting key b0c8f388880e55c2df6d387d8d2a946b
I20260928 15:57:23.711176 27196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb: Bootstrap starting.
I20260928 15:57:23.715302 27196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:23.716615 27196 log.cc:824] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:23.719846 27196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb: No bootstrap required, opened a new log
Sep 28 15:57:23 dist-test-slave-9cqd krb5kdc[27117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:23.728398 27196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42567 } }
I20260928 15:57:23.728791 27196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:23.728914 27196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195b5d3e7b4a4500b33c0bbafe0d4feb, State: Initialized, Role: FOLLOWER
I20260928 15:57:23.729547 27196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42567 } }
I20260928 15:57:23.729843 27196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:23.729998 27196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:23.730186 27196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:23.730427 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:23.708237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.22.125.254:42567)
0928 15:57:23.709066 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:23.709078 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:23.709121 (+ 43us) server_negotiation.cc:407] Connection header received
0928 15:57:23.710776 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:23.710798 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:23.711171 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:23.711524 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:23.713473 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:23.715344 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:23.717226 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:23.718202 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:23.718335 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:23.722906 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:23.722914 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:23.722930 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:23.722973 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:23.726087 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:23.726764 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:23.726771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:23.726776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:23.726941 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:23.727359 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:23.727366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:23.727368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:23.727860 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:23.728188 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:23.728629 (+ 441us) server_negotiation.cc:299] Negotiation successful
0928 15:57:23.729221 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":398,"threads_started":1}
I20260928 15:57:23.733485 27196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42567 } }
I20260928 15:57:23.734009 27196 leader_election.cc:302] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 195b5d3e7b4a4500b33c0bbafe0d4feb; no voters:
I20260928 15:57:23.734602 27196 leader_election.cc:288] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:23.734720 27201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:23.736588 27196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:23.741361 27201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [term 1 LEADER]: Becoming Leader. State: Replica: 195b5d3e7b4a4500b33c0bbafe0d4feb, State: Running, Role: LEADER
I20260928 15:57:23.741993 27201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42567 } }
I20260928 15:57:23.744930 27202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42567 } } }
I20260928 15:57:23.745188 27202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:23.746438 27215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:23.746712 27203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 195b5d3e7b4a4500b33c0bbafe0d4feb. Latest consensus state: current_term: 1 leader_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "195b5d3e7b4a4500b33c0bbafe0d4feb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42567 } } }
I20260928 15:57:23.747150 27203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:23.751590 27215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:23.759598 27215 catalog_manager.cc:1347] Generated new cluster ID: 377f354190ba404f9206d8f0ca3012e5
I20260928 15:57:23.759716 27215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:23.771417 27215 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:23.772723 27215 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:23.783039 27215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb: Generated new TSK 0
I20260928 15:57:23.783762 27215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:57:23.829322 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42567
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:24.146270 27222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:24.146566 27222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:24.146729 27222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:24.154126 27222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:24.154253 27222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:24.154445 27222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:24.163904 27222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:24.165126 27222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:24.166414 27222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:24.178566 27230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:24.178534 27227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:24.178596 27228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:24.179939 27222 server_base.cc:1034] running on GCE node
I20260928 15:57:24.180523 27222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:24.181408 27222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:24.182623 27222 hybrid_clock.cc:648] HybridClock initialized: now 1790611044182585 us; error 52 us; skew 500 ppm
I20260928 15:57:24.182926 27222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:24 dist-test-slave-9cqd krb5kdc[27117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611044, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:24.185930 27222 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:57:24.187009 27222 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:24.187098 27222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:24.187361 27222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:24.190157 27222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "6642f0d49b9b49e691666317bd2a27b1"
format_stamp: "Formatted at 2026-09-28 15:57:24 on dist-test-slave-9cqd"
server_key: "20a3750f723ee8f88c40ae152b823210"
server_key_iv: "012967f81e448226a4b43e77c0cb47bf"
server_key_version: "encryptionkey@0"
I20260928 15:57:24.190886 27222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "6642f0d49b9b49e691666317bd2a27b1"
format_stamp: "Formatted at 2026-09-28 15:57:24 on dist-test-slave-9cqd"
server_key: "20a3750f723ee8f88c40ae152b823210"
server_key_iv: "012967f81e448226a4b43e77c0cb47bf"
server_key_version: "encryptionkey@0"
I20260928 15:57:24.195747 27222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 15:57:24.198976 27236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:24.200287 27222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:24.200435 27222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "6642f0d49b9b49e691666317bd2a27b1"
format_stamp: "Formatted at 2026-09-28 15:57:24 on dist-test-slave-9cqd"
server_key: "20a3750f723ee8f88c40ae152b823210"
server_key_iv: "012967f81e448226a4b43e77c0cb47bf"
server_key_version: "encryptionkey@0"
I20260928 15:57:24.200604 27222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:24.214630 27222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:24.218257 27222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:24.218478 27222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:24.219265 27222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:24.220491 27222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:24.220577 27222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:24.220650 27222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:24.220705 27222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:24.250725 27222 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:39039
I20260928 15:57:24.250741 27348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:39039 every 8 connection(s)
I20260928 15:57:24.251619 27222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:24.256893 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 27222
I20260928 15:57:24.257084 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:24.257709 23031 external_mini_cluster.cc:1428] Setting key 0a895f255814c2d2a66a843f01a8183a
Sep 28 15:57:24 dist-test-slave-9cqd krb5kdc[27117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611044, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:24.272186 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:24.254178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:57171 (local address 127.22.125.254:42567)
0928 15:57:24.254601 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:24.254605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:24.255546 (+ 941us) server_negotiation.cc:407] Connection header received
0928 15:57:24.257451 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:24.257456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:24.257547 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:24.257662 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:24.259094 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.259867 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.261424 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.261808 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.261943 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:24.265321 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:24.265330 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:24.265338 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:24.265383 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:24.269008 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:24.269666 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:24.269674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:24.269681 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:24.269829 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:24.270226 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:24.270234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:24.270240 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:24.270622 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:24.270776 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:24.271401 (+ 625us) server_negotiation.cc:299] Negotiation successful
0928 15:57:24.271649 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":217,"threads_started":1}
I20260928 15:57:24.272756 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:24.254737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42567 (local address 127.22.125.193:57171)
0928 15:57:24.255403 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:24.255439 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:24.257201 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:24.257892 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:24.257900 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:24.258397 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:24.258925 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:24.258941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.260019 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.260023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:24.261252 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:24.261264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.262032 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.262041 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:24.262283 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:24.262939 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:24.262965 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:24.265114 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:24.269131 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:24.269137 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:24.269147 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:24.269534 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:24.269943 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:24.269946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:24.269948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:24.270114 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:24.270747 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:24.270752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:24.271038 (+ 286us) client_negotiation.cc:769] Sending connection context
0928 15:57:24.271244 (+ 206us) client_negotiation.cc:240] Negotiation successful
0928 15:57:24.271581 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":261,"threads_started":1}
I20260928 15:57:24.274322 27349 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42567
I20260928 15:57:24.274658 27349 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:24.275472 27349 heartbeater.cc:511] Master 127.22.125.254:42567 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:57:24.277899 27161 ts_manager.cc:194] Registered new tserver with Master: 6642f0d49b9b49e691666317bd2a27b1 (127.22.125.193:39039)
I20260928 15:57:24.279290 27161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:57171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:57:24.300992 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42567
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:24.614516 27355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:24.614779 27355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:24.614877 27355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:24.622193 27355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:24.622289 27355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:24.622411 27355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:24.631728 27355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:24.632916 27355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:24.634099 27355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:24.647672 27363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:24.647728 27361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:24.647748 27360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:24.648731 27355 server_base.cc:1034] running on GCE node
I20260928 15:57:24.649310 27355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:24.650185 27355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:24.651407 27355 hybrid_clock.cc:648] HybridClock initialized: now 1790611044651364 us; error 37 us; skew 500 ppm
I20260928 15:57:24.651711 27355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:24 dist-test-slave-9cqd krb5kdc[27117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611044, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:24.654614 27355 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:57:24.655731 27355 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:24.655861 27355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:24.656220 27355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:24.658965 27355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "c585f921e2fa4ad58515fd820429c315"
format_stamp: "Formatted at 2026-09-28 15:57:24 on dist-test-slave-9cqd"
server_key: "f4d4e0debb537b3562ad88c29c108320"
server_key_iv: "e0a33c25ceac0d55a7fedab93bc5db11"
server_key_version: "encryptionkey@0"
I20260928 15:57:24.659750 27355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "c585f921e2fa4ad58515fd820429c315"
format_stamp: "Formatted at 2026-09-28 15:57:24 on dist-test-slave-9cqd"
server_key: "f4d4e0debb537b3562ad88c29c108320"
server_key_iv: "e0a33c25ceac0d55a7fedab93bc5db11"
server_key_version: "encryptionkey@0"
I20260928 15:57:24.664523 27355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 15:57:24.667681 27369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:24.668929 27355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 15:57:24.669097 27355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "c585f921e2fa4ad58515fd820429c315"
format_stamp: "Formatted at 2026-09-28 15:57:24 on dist-test-slave-9cqd"
server_key: "f4d4e0debb537b3562ad88c29c108320"
server_key_iv: "e0a33c25ceac0d55a7fedab93bc5db11"
server_key_version: "encryptionkey@0"
I20260928 15:57:24.669309 27355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:24.690686 27355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:24.694483 27355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:24.694746 27355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:24.695739 27355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:24.697300 27355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:24.697433 27355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:24.697549 27355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:24.697639 27355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:24.734454 27355 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:36857
I20260928 15:57:24.734508 27481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:36857 every 8 connection(s)
I20260928 15:57:24.735303 27355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:24.737808 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 27355
I20260928 15:57:24.738005 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:24.738592 23031 external_mini_cluster.cc:1428] Setting key defecaf49179511f4887a2e8b63aa90a
Sep 28 15:57:24 dist-test-slave-9cqd krb5kdc[27117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611044, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:24.755618 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:24.737665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:43791 (local address 127.22.125.254:42567)
0928 15:57:24.737836 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:24.737848 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:24.740081 (+ 2233us) server_negotiation.cc:407] Connection header received
0928 15:57:24.741907 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:24.741913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:24.741997 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:24.742101 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:24.743535 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.744322 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.745821 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.746235 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.746376 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:24.750037 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:24.750042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:24.750045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:24.750074 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:24.752731 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:24.753430 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:24.753437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:24.753440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:24.753572 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:24.753971 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:24.753978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:24.753983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:24.754240 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:24.754411 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:24.755059 (+ 648us) server_negotiation.cc:299] Negotiation successful
0928 15:57:24.755279 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 15:57:24.756608 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:24.738375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42567 (local address 127.22.125.194:43791)
0928 15:57:24.739896 (+ 1521us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:24.739948 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:24.741654 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:24.742319 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:24.742327 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:24.742809 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:24.743352 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:24.743373 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.744467 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.744473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:24.745676 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:24.745688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:24.746452 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:24.746460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:24.746684 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:24.747377 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:24.747432 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:24.749823 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:24.752860 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:24.752868 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:24.752883 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:24.753284 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:24.753681 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:24.753686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:24.753690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:24.753863 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:24.754377 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:24.754385 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:24.754710 (+ 325us) client_negotiation.cc:769] Sending connection context
0928 15:57:24.754963 (+ 253us) client_negotiation.cc:240] Negotiation successful
0928 15:57:24.755487 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1241,"thread_start_us":1016,"threads_started":1}
I20260928 15:57:24.758024 27482 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42567
I20260928 15:57:24.758338 27482 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:24.759035 27482 heartbeater.cc:511] Master 127.22.125.254:42567 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
I20260928 15:57:24.760622 27161 ts_manager.cc:194] Registered new tserver with Master: c585f921e2fa4ad58515fd820429c315 (127.22.125.194:36857)
I20260928 15:57:24.761423 27161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:43791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:57:24.782604 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42567
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:25.095983 27487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:25.096349 27487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:25.096536 27487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:25.103487 27487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:25.103580 27487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:25.103798 27487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:25.113363 27487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:25.114576 27487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:25.115849 27487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:25.128465 27493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:25.128820 27492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:25.128554 27495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:25.129598 27487 server_base.cc:1034] running on GCE node
I20260928 15:57:25.130254 27487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:25.131145 27487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:25.132360 27487 hybrid_clock.cc:648] HybridClock initialized: now 1790611045132319 us; error 52 us; skew 500 ppm
I20260928 15:57:25.132668 27487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:25 dist-test-slave-9cqd krb5kdc[27117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611045, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:25.135619 27487 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:57:25.136770 27487 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:25.136865 27487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:25.137192 27487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:25.140219 27487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e"
format_stamp: "Formatted at 2026-09-28 15:57:25 on dist-test-slave-9cqd"
server_key: "0e2e96a407fb053164671ebb4f550b40"
server_key_iv: "ec7b02d9bee5c53f97d96610bf47fecf"
server_key_version: "encryptionkey@0"
I20260928 15:57:25.140960 27487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e"
format_stamp: "Formatted at 2026-09-28 15:57:25 on dist-test-slave-9cqd"
server_key: "0e2e96a407fb053164671ebb4f550b40"
server_key_iv: "ec7b02d9bee5c53f97d96610bf47fecf"
server_key_version: "encryptionkey@0"
I20260928 15:57:25.145632 27487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260928 15:57:25.149219 27501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:25.150517 27487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:25.150642 27487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e"
format_stamp: "Formatted at 2026-09-28 15:57:25 on dist-test-slave-9cqd"
server_key: "0e2e96a407fb053164671ebb4f550b40"
server_key_iv: "ec7b02d9bee5c53f97d96610bf47fecf"
server_key_version: "encryptionkey@0"
I20260928 15:57:25.150877 27487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:25.170420 27487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:25.174073 27487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:25.174283 27487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:25.175150 27487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:25.176613 27487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:25.176723 27487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:25.176847 27487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:25.176928 27487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:25.209455 27487 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:38301
I20260928 15:57:25.209534 27613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:38301 every 8 connection(s)
I20260928 15:57:25.210285 27487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:25.220551 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 27487
I20260928 15:57:25.220728 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:25.221131 23031 external_mini_cluster.cc:1428] Setting key 2404bc8e2dd12f1b4e4d3491657f216a
Sep 28 15:57:25 dist-test-slave-9cqd krb5kdc[27117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611045, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:25.228785 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.212694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:56453 (local address 127.22.125.254:42567)
0928 15:57:25.212873 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.212877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.213979 (+ 1102us) server_negotiation.cc:407] Connection header received
0928 15:57:25.215712 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.215718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.215804 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.215912 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:25.217388 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.218140 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.219643 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.220099 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.220274 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.223913 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:25.223919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:25.223923 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:25.223963 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:25.226345 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.226970 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.226974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.226976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.227084 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.227421 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.227424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.227425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.227669 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:25.227774 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.228414 (+ 640us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.228561 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 15:57:25.229664 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.213153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42567 (local address 127.22.125.195:56453)
0928 15:57:25.213816 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.213867 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:25.215452 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:25.216160 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:25.216168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:25.216649 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:25.217187 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.217203 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.218293 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.218298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.219497 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.219508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.220360 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.220369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.220604 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.221316 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:25.221337 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:25.223683 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:25.226479 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:25.226485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:25.226496 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:25.226877 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:25.227168 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:25.227171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:25.227173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:25.227328 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:25.227771 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:25.227776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:25.228077 (+ 301us) client_negotiation.cc:769] Sending connection context
0928 15:57:25.228308 (+ 231us) client_negotiation.cc:240] Negotiation successful
0928 15:57:25.228651 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":277,"threads_started":1}
I20260928 15:57:25.230923 27614 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42567
I20260928 15:57:25.231164 27614 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:25.231715 27614 heartbeater.cc:511] Master 127.22.125.254:42567 requested a full tablet report, sending...
I20260928 15:57:25.233027 27161 ts_manager.cc:194] Registered new tserver with Master: 90c5f2ebfcb44a6b82b9bdec76dd156e (127.22.125.195:38301)
I20260928 15:57:25.233737 27161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:56453
I20260928 15:57:25.236011 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:57:25.252315 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.240558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39112 (local address 127.22.125.254:42567)
0928 15:57:25.240749 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.240756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.241174 (+ 418us) server_negotiation.cc:407] Connection header received
0928 15:57:25.241286 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.241289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.241340 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.241420 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:25.242919 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.243605 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.244820 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.245228 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.245315 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.247331 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:25.247335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:25.247338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:25.247367 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:25.249634 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.250234 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.250239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.250241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.250428 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.250794 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.250799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.250802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.251244 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:25.251508 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.251756 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.251974 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 15:57:25.256259 27161 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:25.262797 27161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:25.282186 27349 heartbeater.cc:503] Master 127.22.125.254:42567 was elected leader, sending a full tablet report...
I20260928 15:57:25.285187 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.274622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.22.125.193:39039)
0928 15:57:25.275309 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.275318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.276061 (+ 743us) server_negotiation.cc:407] Connection header received
0928 15:57:25.276474 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.276486 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.276748 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.276944 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:25.277990 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.279554 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.282410 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.283987 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.284222 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.284397 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.284688 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.284849 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":382,"threads_started":1}
I20260928 15:57:25.285238 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.274547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60056)
0928 15:57:25.275892 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.275931 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:25.276208 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:25.277265 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:25.277277 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:25.277326 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:25.277765 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.277780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.279724 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.279732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.282207 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.282224 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.284213 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.284220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.284421 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.284480 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 15:57:25.284622 (+ 142us) client_negotiation.cc:240] Negotiation successful
0928 15:57:25.284812 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"mutex_wait_us":61,"spinlock_wait_cycles":6912,"thread_start_us":2296,"threads_started":1}
I20260928 15:57:25.286376 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.275193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.22.125.194:36857)
0928 15:57:25.276711 (+ 1518us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.276720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.277225 (+ 505us) server_negotiation.cc:407] Connection header received
0928 15:57:25.277556 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.277563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.277792 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.277951 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:25.278950 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.280318 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.283556 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.284949 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.285106 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.285302 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.285843 (+ 541us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.286048 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376,"thread_start_us":2270,"threads_started":1}
I20260928 15:57:25.286595 27630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.276517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:38301 (local address 127.0.0.1:58478)
0928 15:57:25.277425 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.277444 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:25.277572 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:25.279260 (+ 1688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:25.279264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:25.279290 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:25.279623 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.279634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.281598 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.281604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.283641 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.283656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.285728 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.285733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.285915 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.285946 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 15:57:25.286038 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 15:57:25.286180 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":1,"thread_start_us":1114,"threads_started":1}
I20260928 15:57:25.287377 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.274634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:36857 (local address 127.0.0.1:32818)
0928 15:57:25.277121 (+ 2487us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.277139 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:25.277329 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:25.278280 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:25.278284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:25.278310 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:25.278752 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.278764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.280463 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.280472 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.283363 (+ 2891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.283378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.285463 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.285469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.285647 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.285688 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 15:57:25.285776 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 15:57:25.285911 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2355,"mutex_wait_us":66,"spinlock_wait_cycles":4224,"thread_start_us":880,"threads_started":1}
I20260928 15:57:25.287452 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.277203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.22.125.195:38301)
0928 15:57:25.278454 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.278464 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.278496 (+ 32us) server_negotiation.cc:407] Connection header received
0928 15:57:25.278586 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.278595 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.278815 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.278996 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:25.279836 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.281450 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.283824 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.285298 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.285500 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.285612 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.286952 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.287129 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1072,"thread_start_us":900,"threads_started":1}
I20260928 15:57:25.289575 27284 tablet_service.cc:1467] Processing CreateTablet for tablet 82afac58845448a8a91d0fe715f91556 (DEFAULT_TABLE table=test-table [id=98b258450e12493c916e9e36a4a9e76c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:25.290764 27284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82afac58845448a8a91d0fe715f91556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:25.290882 27549 tablet_service.cc:1467] Processing CreateTablet for tablet 82afac58845448a8a91d0fe715f91556 (DEFAULT_TABLE table=test-table [id=98b258450e12493c916e9e36a4a9e76c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:25.290882 27417 tablet_service.cc:1467] Processing CreateTablet for tablet 82afac58845448a8a91d0fe715f91556 (DEFAULT_TABLE table=test-table [id=98b258450e12493c916e9e36a4a9e76c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:25.292250 27549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82afac58845448a8a91d0fe715f91556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:25.292415 27417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82afac58845448a8a91d0fe715f91556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:25.301741 27632 tablet_bootstrap.cc:492] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: Bootstrap starting.
I20260928 15:57:25.305776 27632 tablet_bootstrap.cc:654] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:25.306797 27632 log.cc:824] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:25.308221 27633 tablet_bootstrap.cc:492] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Bootstrap starting.
I20260928 15:57:25.309007 27634 tablet_bootstrap.cc:492] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Bootstrap starting.
I20260928 15:57:25.310014 27632 tablet_bootstrap.cc:492] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: No bootstrap required, opened a new log
I20260928 15:57:25.310287 27632 ts_tablet_manager.cc:1397] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260928 15:57:25.312593 27634 tablet_bootstrap.cc:654] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:25.313819 27634 log.cc:824] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:25.315346 27633 tablet_bootstrap.cc:654] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:25.316284 27633 log.cc:824] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:25.316936 27634 tablet_bootstrap.cc:492] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: No bootstrap required, opened a new log
I20260928 15:57:25.317196 27634 ts_tablet_manager.cc:1397] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 15:57:25.318593 27633 tablet_bootstrap.cc:492] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: No bootstrap required, opened a new log
I20260928 15:57:25.318832 27633 ts_tablet_manager.cc:1397] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260928 15:57:25.320600 27632 raft_consensus.cc:348] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.321041 27632 raft_consensus.cc:374] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:25.321141 27632 raft_consensus.cc:729] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6642f0d49b9b49e691666317bd2a27b1, State: Initialized, Role: FOLLOWER
I20260928 15:57:25.321734 27632 consensus_queue.cc:260] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.326524 27633 raft_consensus.cc:348] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.326864 27633 raft_consensus.cc:374] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:25.327015 27633 raft_consensus.cc:729] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c585f921e2fa4ad58515fd820429c315, State: Initialized, Role: FOLLOWER
I20260928 15:57:25.327667 27633 consensus_queue.cc:260] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.328990 27632 ts_tablet_manager.cc:1428] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: Time spent starting tablet: real 0.019s user 0.019s sys 0.000s
I20260928 15:57:25.328979 27634 raft_consensus.cc:348] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.329466 27634 raft_consensus.cc:374] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:25.329610 27634 raft_consensus.cc:729] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c5f2ebfcb44a6b82b9bdec76dd156e, State: Initialized, Role: FOLLOWER
I20260928 15:57:25.329986 27482 heartbeater.cc:503] Master 127.22.125.254:42567 was elected leader, sending a full tablet report...
I20260928 15:57:25.330374 27634 consensus_queue.cc:260] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.330639 27633 ts_tablet_manager.cc:1428] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260928 15:57:25.331681 27614 heartbeater.cc:503] Master 127.22.125.254:42567 was elected leader, sending a full tablet report...
I20260928 15:57:25.332350 27634 ts_tablet_manager.cc:1428] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260928 15:57:25.352346 27640 raft_consensus.cc:482] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:25.352610 27640 raft_consensus.cc:504] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.354287 27640 leader_election.cc:288] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6642f0d49b9b49e691666317bd2a27b1 (127.22.125.193:39039), c585f921e2fa4ad58515fd820429c315 (127.22.125.194:36857)
I20260928 15:57:25.360410 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.354635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:48507 (local address 127.22.125.193:39039)
0928 15:57:25.354824 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.354831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.354852 (+ 21us) server_negotiation.cc:407] Connection header received
0928 15:57:25.354987 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.354993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.355083 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.355182 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:25.355658 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.356684 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.358515 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.359612 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.359728 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.359777 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.360145 (+ 368us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.360237 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 15:57:25.360684 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.354407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.22.125.195:48507)
0928 15:57:25.354717 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.354740 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:25.354853 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:25.355243 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:25.355248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:25.355288 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:25.355522 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.355538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.356815 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.356821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.358363 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.358375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.359755 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.359761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.359959 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.359996 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 15:57:25.360107 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 15:57:25.360242 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":2}
I20260928 15:57:25.361384 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.354836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:54551 (local address 127.22.125.194:36857)
0928 15:57:25.355053 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.355062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.355535 (+ 473us) server_negotiation.cc:407] Connection header received
0928 15:57:25.355723 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.355728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.355814 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.355890 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:25.356482 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.357455 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.359429 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.360489 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.360642 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.360724 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.361071 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.361197 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 15:57:25.361454 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.354720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:36857 (local address 127.22.125.195:54551)
0928 15:57:25.355444 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.355456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:25.355580 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:25.355923 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:25.355930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:25.355955 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:25.356336 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.356348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.357600 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.357604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.359274 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:25.359289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.360714 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.360719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:25.360913 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.360942 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 15:57:25.361016 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 15:57:25.361095 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":3,"thread_start_us":416,"threads_started":1}
I20260928 15:57:25.361718 27304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82afac58845448a8a91d0fe715f91556" candidate_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6642f0d49b9b49e691666317bd2a27b1" is_pre_election: true
I20260928 15:57:25.362329 27304 raft_consensus.cc:2454] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c5f2ebfcb44a6b82b9bdec76dd156e in term 0.
I20260928 15:57:25.362390 27437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82afac58845448a8a91d0fe715f91556" candidate_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c585f921e2fa4ad58515fd820429c315" is_pre_election: true
I20260928 15:57:25.362846 27437 raft_consensus.cc:2454] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c5f2ebfcb44a6b82b9bdec76dd156e in term 0.
I20260928 15:57:25.363025 27503 leader_election.cc:302] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6642f0d49b9b49e691666317bd2a27b1, 90c5f2ebfcb44a6b82b9bdec76dd156e; no voters:
I20260928 15:57:25.363394 27640 raft_consensus.cc:2781] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:25.363549 27640 raft_consensus.cc:482] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:25.363656 27640 raft_consensus.cc:3037] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:25.365734 27640 raft_consensus.cc:504] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.366308 27640 leader_election.cc:288] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [CANDIDATE]: Term 1 election: Requested vote from peers 6642f0d49b9b49e691666317bd2a27b1 (127.22.125.193:39039), c585f921e2fa4ad58515fd820429c315 (127.22.125.194:36857)
I20260928 15:57:25.366685 27304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82afac58845448a8a91d0fe715f91556" candidate_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6642f0d49b9b49e691666317bd2a27b1"
I20260928 15:57:25.366737 27437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82afac58845448a8a91d0fe715f91556" candidate_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c585f921e2fa4ad58515fd820429c315"
I20260928 15:57:25.366899 27304 raft_consensus.cc:3037] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:25.366901 27437 raft_consensus.cc:3037] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:25.370075 27437 raft_consensus.cc:2454] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c5f2ebfcb44a6b82b9bdec76dd156e in term 1.
I20260928 15:57:25.370075 27304 raft_consensus.cc:2454] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c5f2ebfcb44a6b82b9bdec76dd156e in term 1.
I20260928 15:57:25.370455 27503 leader_election.cc:302] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6642f0d49b9b49e691666317bd2a27b1, 90c5f2ebfcb44a6b82b9bdec76dd156e; no voters:
I20260928 15:57:25.370715 27640 raft_consensus.cc:2781] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:25.371171 27640 raft_consensus.cc:686] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 1 LEADER]: Becoming Leader. State: Replica: 90c5f2ebfcb44a6b82b9bdec76dd156e, State: Running, Role: LEADER
I20260928 15:57:25.371721 27640 consensus_queue.cc:237] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } }
I20260928 15:57:25.376549 27159 catalog_manager.cc:5495] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e reported cstate change: term changed from 0 to 1, leader changed from <none> to 90c5f2ebfcb44a6b82b9bdec76dd156e (127.22.125.195). New cstate: current_term: 1 leader_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c5f2ebfcb44a6b82b9bdec76dd156e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38301 } health_report { overall_health: HEALTHY } } }
I20260928 15:57:25.408339 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.402965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.22.125.195:38301)
0928 15:57:25.403216 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.403222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.403316 (+ 94us) server_negotiation.cc:407] Connection header received
0928 15:57:25.403551 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.403555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.403659 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.403802 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:25.404172 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.404878 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.406275 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.406670 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.406799 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.407255 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:25.407745 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:25.407897 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.407977 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.408076 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 15:57:25.416189 27437 raft_consensus.cc:1261] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 1 FOLLOWER]: Refusing update from remote peer 90c5f2ebfcb44a6b82b9bdec76dd156e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:25.416229 27304 raft_consensus.cc:1261] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 1 FOLLOWER]: Refusing update from remote peer 90c5f2ebfcb44a6b82b9bdec76dd156e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:25.417088 27642 consensus_queue.cc:1035] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c585f921e2fa4ad58515fd820429c315" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 36857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:25.417402 27640 consensus_queue.cc:1035] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6642f0d49b9b49e691666317bd2a27b1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:25.442224 27159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:25.442518 27159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:25.445950 27159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 195b5d3e7b4a4500b33c0bbafe0d4feb: Sending DeleteTablet for 3 replicas of tablet 82afac58845448a8a91d0fe715f91556
I20260928 15:57:25.446985 27284 tablet_service.cc:1514] Processing DeleteTablet for tablet 82afac58845448a8a91d0fe715f91556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:25 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:60056
I20260928 15:57:25.447140 27417 tablet_service.cc:1514] Processing DeleteTablet for tablet 82afac58845448a8a91d0fe715f91556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:25 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:32818
I20260928 15:57:25.447649 27549 tablet_service.cc:1514] Processing DeleteTablet for tablet 82afac58845448a8a91d0fe715f91556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:25 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:58478
I20260928 15:57:25.448423 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 27222
I20260928 15:57:25.472685 27657 tablet_replica.cc:330] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: stopping tablet replica
I20260928 15:57:25.473430 27657 raft_consensus.cc:2229] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 1 LEADER]: Raft consensus shutting down.
Sep 28 15:57:25 dist-test-slave-9cqd krb5kdc[27117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:25.474845 27655 tablet_replica.cc:330] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: stopping tablet replica
I20260928 15:57:25.475045 27657 raft_consensus.cc:2258] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:25.475983 27655 raft_consensus.cc:2229] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:25.477970 27657 ts_tablet_manager.cc:1905] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:25.478521 27655 raft_consensus.cc:2258] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:25.483233 27655 ts_tablet_manager.cc:1905] T 82afac58845448a8a91d0fe715f91556 P 6642f0d49b9b49e691666317bd2a27b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:25.485661 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.450072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.22.125.193:39039)
0928 15:57:25.450298 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.450307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.460681 (+ 10374us) server_negotiation.cc:407] Connection header received
0928 15:57:25.461204 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.461214 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.461323 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.461429 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:25.463717 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.465288 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.467851 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.468821 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.468947 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.475858 (+ 6911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:25.475879 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:25.475904 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:25.475965 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:25.481065 (+ 5100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.482163 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.482175 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.482182 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.482401 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.482949 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.482957 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.482963 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.483532 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:25.483913 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.484713 (+ 800us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.485047 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 15:57:25.488222 27344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:25.489514 27656 tablet_replica.cc:330] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: stopping tablet replica
I20260928 15:57:25.491901 27656 raft_consensus.cc:2229] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:25.492408 27656 raft_consensus.cc:2258] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:25.494252 27656 ts_tablet_manager.cc:1905] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:25.494519 27657 ts_tablet_manager.cc:1918] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:25.494767 27657 log.cc:1196] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/82afac58845448a8a91d0fe715f91556
I20260928 15:57:25.495399 27657 ts_tablet_manager.cc:1939] T 82afac58845448a8a91d0fe715f91556 P 90c5f2ebfcb44a6b82b9bdec76dd156e: Deleting consensus metadata
I20260928 15:57:25.497304 27145 catalog_manager.cc:4841] TS 90c5f2ebfcb44a6b82b9bdec76dd156e (127.22.125.195:38301): tablet 82afac58845448a8a91d0fe715f91556 (table test-table [id=98b258450e12493c916e9e36a4a9e76c]) successfully deleted
I20260928 15:57:25.501648 27656 ts_tablet_manager.cc:1918] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:25.501827 27656 log.cc:1196] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/82afac58845448a8a91d0fe715f91556
I20260928 15:57:25.502419 27656 ts_tablet_manager.cc:1939] T 82afac58845448a8a91d0fe715f91556 P c585f921e2fa4ad58515fd820429c315: Deleting consensus metadata
I20260928 15:57:25.504323 27145 catalog_manager.cc:4841] TS c585f921e2fa4ad58515fd820429c315 (127.22.125.194:36857): tablet 82afac58845448a8a91d0fe715f91556 (table test-table [id=98b258450e12493c916e9e36a4a9e76c]) successfully deleted
I20260928 15:57:25.585417 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 27222
I20260928 15:57:25.602149 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 27355
W20260928 15:57:25.602149 27146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.125.193:39039 (error 108)
I20260928 15:57:25.604408 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.603848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60074)
0928 15:57:25.604016 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.604226 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 15:57:25.605468 27146 catalog_manager.cc:4584] TS 6642f0d49b9b49e691666317bd2a27b1 (127.22.125.193:39039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82afac58845448a8a91d0fe715f91556: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Sep 28 15:57:25 dist-test-slave-9cqd krb5kdc[27117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:57:25.618800 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.603443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32822 (local address 127.22.125.194:36857)
0928 15:57:25.603603 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.603609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.603628 (+ 19us) server_negotiation.cc:407] Connection header received
0928 15:57:25.603724 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.603729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.603809 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.603911 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:25.605400 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.606581 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.607749 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.608401 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.608528 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.612353 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:25.612371 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:25.612390 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:25.612431 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:25.616003 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.616761 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.616771 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.616774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.616923 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.617248 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.617253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.617256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.617722 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:25.618017 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.618206 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.618432 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 15:57:25.619467 27477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:25.659330 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.658775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60080)
0928 15:57:25.659015 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.659133 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260928 15:57:25.660552 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.660134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60092)
0928 15:57:25.660291 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.660402 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260928 15:57:25.708986 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 27355
I20260928 15:57:25.721326 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.720648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60104)
0928 15:57:25.720974 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.721090 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260928 15:57:25.722757 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.722235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60106)
0928 15:57:25.722495 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.722586 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260928 15:57:25.726997 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 27487
Sep 28 15:57:25 dist-test-slave-9cqd krb5kdc[27117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:57:25.744098 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.727704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58502 (local address 127.22.125.195:38301)
0928 15:57:25.727904 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:25.727910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:25.727933 (+ 23us) server_negotiation.cc:407] Connection header received
0928 15:57:25.728007 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:25.728012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:25.728122 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:25.728253 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:25.729483 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.730784 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.732013 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:25.732719 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:25.732861 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:25.737766 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:25.737778 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:25.737792 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:25.737830 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:25.741210 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.741993 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.741999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.742001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.742148 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:25.742572 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:25.742575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:25.742577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:25.742926 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:25.743148 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:25.743516 (+ 368us) server_negotiation.cc:299] Negotiation successful
0928 15:57:25.743691 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 15:57:25.744819 27609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:25.803383 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.802833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60114)
0928 15:57:25.803081 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.803198 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260928 15:57:25.804510 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:25.804029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39039 (local address 127.0.0.1:60116)
0928 15:57:25.804261 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:25.804384 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260928 15:57:25.867745 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 27487
I20260928 15:57:25.888304 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 27130
I20260928 15:57:25.889345 27191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:25.996929 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 27130
2026-09-28T15:57:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4000 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T15:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:26Z Disabled control of system clock
I20260928 15:57:26.061618 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:41875
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:41875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 15:57:26.373754 27667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:26.374042 27667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:26.374126 27667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:26.381387 27667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:26.381500 27667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:26.381546 27667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:26.381601 27667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:26.381644 27667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:26.391008 27667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:41875
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:41875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:26.392309 27667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:26.394002 27667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:26.405791 27673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:26.405834 27675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:26.405862 27672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:26.406493 27667 server_base.cc:1034] running on GCE node
I20260928 15:57:26.407271 27667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:26.408602 27667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:26.409796 27667 hybrid_clock.cc:648] HybridClock initialized: now 1790611046409765 us; error 27 us; skew 500 ppm
I20260928 15:57:26.410107 27667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:26.418264 27667 webserver.cc:460] Webserver started at http://127.22.125.254:39645/ using document root <none> and password file <none>
I20260928 15:57:26.418912 27667 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:26.418996 27667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:26.419256 27667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:26.422043 27667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "042246b9cbdf4174984261a9fe87bf2a"
format_stamp: "Formatted at 2026-09-28 15:57:26 on dist-test-slave-9cqd"
server_key: "d1421b960b96cb0b7296b469aee61344"
server_key_iv: "146b0ffa2973fa644d7d7cbd1d4c0289"
server_key_version: "encryptionkey@0"
I20260928 15:57:26.422745 27667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "042246b9cbdf4174984261a9fe87bf2a"
format_stamp: "Formatted at 2026-09-28 15:57:26 on dist-test-slave-9cqd"
server_key: "d1421b960b96cb0b7296b469aee61344"
server_key_iv: "146b0ffa2973fa644d7d7cbd1d4c0289"
server_key_version: "encryptionkey@0"
I20260928 15:57:26.427575 27667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 15:57:26.430753 27681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:26.431988 27667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:26.432173 27667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "042246b9cbdf4174984261a9fe87bf2a"
format_stamp: "Formatted at 2026-09-28 15:57:26 on dist-test-slave-9cqd"
server_key: "d1421b960b96cb0b7296b469aee61344"
server_key_iv: "146b0ffa2973fa644d7d7cbd1d4c0289"
server_key_version: "encryptionkey@0"
I20260928 15:57:26.432369 27667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:26.437721 27667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:26.438398 27667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:26.438640 27667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:26.456153 27667 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:41875
I20260928 15:57:26.456188 27732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:41875 every 8 connection(s)
I20260928 15:57:26.457415 27667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:26.461090 27733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:26.467490 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 27667
I20260928 15:57:26.467713 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:26.468389 23031 external_mini_cluster.cc:1428] Setting key fb6831bc21bce12158bc9e4384cc396e
I20260928 15:57:26.472033 27733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a: Bootstrap starting.
I20260928 15:57:26.476054 27733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:26.477412 27733 log.cc:824] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:26.478051 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:26.470365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.22.125.254:41875)
0928 15:57:26.471412 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:26.471428 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:26.471480 (+ 52us) server_negotiation.cc:407] Connection header received
0928 15:57:26.473896 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:26.473926 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:26.474486 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:26.474917 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:26.475930 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:26.475940 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:26.475957 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:26.475976 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:26.476098 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:26.476125 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:26.476327 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:26.476482 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 15:57:26.476780 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":299,"threads_started":1}
I20260928 15:57:26.480660 27733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a: No bootstrap required, opened a new log
I20260928 15:57:26.488685 27733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042246b9cbdf4174984261a9fe87bf2a" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41875 } }
I20260928 15:57:26.489028 27733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:26.489104 27733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042246b9cbdf4174984261a9fe87bf2a, State: Initialized, Role: FOLLOWER
I20260928 15:57:26.489580 27733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042246b9cbdf4174984261a9fe87bf2a" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41875 } }
I20260928 15:57:26.489768 27733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:26.489869 27733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:26.489990 27733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:26.492117 27733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042246b9cbdf4174984261a9fe87bf2a" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41875 } }
I20260928 15:57:26.492571 27733 leader_election.cc:302] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 042246b9cbdf4174984261a9fe87bf2a; no voters:
I20260928 15:57:26.493160 27733 leader_election.cc:288] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:26.493311 27738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:26.494686 27738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [term 1 LEADER]: Becoming Leader. State: Replica: 042246b9cbdf4174984261a9fe87bf2a, State: Running, Role: LEADER
I20260928 15:57:26.495368 27733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:26.495335 27738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042246b9cbdf4174984261a9fe87bf2a" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41875 } }
I20260928 15:57:26.498775 27739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "042246b9cbdf4174984261a9fe87bf2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042246b9cbdf4174984261a9fe87bf2a" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41875 } } }
I20260928 15:57:26.499167 27739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:26.499735 27740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 042246b9cbdf4174984261a9fe87bf2a. Latest consensus state: current_term: 1 leader_uuid: "042246b9cbdf4174984261a9fe87bf2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042246b9cbdf4174984261a9fe87bf2a" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41875 } } }
I20260928 15:57:26.502796 27740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:26.503980 27748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:26.509333 27748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:26.517768 27748 catalog_manager.cc:1347] Generated new cluster ID: b3328121ad9b4e0d849b86cbd0039bcf
I20260928 15:57:26.517879 27748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:26.551419 27748 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:26.551853 27748 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:26.557075 27748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a: Generated new TSK 0
I20260928 15:57:26.557842 27748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 15:57:26.563385 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41875
--builtin_ntp_servers=127.22.125.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 15:57:26.883623 27757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:26.883893 27757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:26.884091 27757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:26.891351 27757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:26.891445 27757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:26.891585 27757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:26.900985 27757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:26.902254 27757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:26.903995 27757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:26.916473 27762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:26.916556 27765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:26.916584 27763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:26.917562 27757 server_base.cc:1034] running on GCE node
I20260928 15:57:26.918144 27757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:26.919004 27757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:26.920213 27757 hybrid_clock.cc:648] HybridClock initialized: now 1790611046920160 us; error 40 us; skew 500 ppm
I20260928 15:57:26.920565 27757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:26.922731 27757 webserver.cc:460] Webserver started at http://127.22.125.193:40819/ using document root <none> and password file <none>
I20260928 15:57:26.923410 27757 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:26.923522 27757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:26.923836 27757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:26.926612 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "f9e13fe0aa804e969118a93e35555211"
format_stamp: "Formatted at 2026-09-28 15:57:26 on dist-test-slave-9cqd"
server_key: "e5a6f5940935fb33981b34b1765c9515"
server_key_iv: "635f4cc46cbf42e1fe8a333e7fd89b28"
server_key_version: "encryptionkey@0"
I20260928 15:57:26.927388 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "f9e13fe0aa804e969118a93e35555211"
format_stamp: "Formatted at 2026-09-28 15:57:26 on dist-test-slave-9cqd"
server_key: "e5a6f5940935fb33981b34b1765c9515"
server_key_iv: "635f4cc46cbf42e1fe8a333e7fd89b28"
server_key_version: "encryptionkey@0"
I20260928 15:57:26.932142 27757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260928 15:57:26.935375 27771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:26.936816 27757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:26.936990 27757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "f9e13fe0aa804e969118a93e35555211"
format_stamp: "Formatted at 2026-09-28 15:57:26 on dist-test-slave-9cqd"
server_key: "e5a6f5940935fb33981b34b1765c9515"
server_key_iv: "635f4cc46cbf42e1fe8a333e7fd89b28"
server_key_version: "encryptionkey@0"
I20260928 15:57:26.937194 27757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:26.942541 27757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:26.943275 27757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:26.943543 27757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:26.944491 27757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:26.945796 27757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:26.945924 27757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:26.946247 27757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:26.946349 27757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:26.978343 27757 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:33013
I20260928 15:57:26.978423 27884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:33013 every 8 connection(s)
I20260928 15:57:26.979725 27757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:26.981616 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 27757
I20260928 15:57:26.981835 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:26.982491 23031 external_mini_cluster.cc:1428] Setting key cf8cdfbe231fd119b2311e9b5c76bf3f
I20260928 15:57:26.985615 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41875
--builtin_ntp_servers=127.22.125.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 15:57:26.988664 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:26.982575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:39043 (local address 127.22.125.254:41875)
0928 15:57:26.983044 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:26.983050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:26.984443 (+ 1393us) server_negotiation.cc:407] Connection header received
0928 15:57:26.986327 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:26.986335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:26.986402 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:26.986525 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:26.987785 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:26.987793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:26.987800 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:26.987822 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:26.987875 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:26.987893 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:26.988015 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:26.988250 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 15:57:26.988318 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":246,"threads_started":1}
I20260928 15:57:26.989493 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:26.983230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:41875 (local address 127.22.125.193:39043)
0928 15:57:26.984283 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:26.984343 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:26.986066 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:26.986698 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:26.986706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:26.986794 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:26.987491 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:26.987493 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:26.987529 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:26.987611 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:26.987993 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:26.988001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:26.988006 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 15:57:26.988274 (+ 268us) client_negotiation.cc:240] Negotiation successful
0928 15:57:26.988460 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":320,"threads_started":1}
I20260928 15:57:26.990988 27885 heartbeater.cc:347] Connected to a master server at 127.22.125.254:41875
I20260928 15:57:26.991259 27885 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:26.991809 27885 heartbeater.cc:511] Master 127.22.125.254:41875 requested a full tablet report, sending...
I20260928 15:57:26.993935 27698 ts_manager.cc:194] Registered new tserver with Master: f9e13fe0aa804e969118a93e35555211 (127.22.125.193:33013)
W20260928 15:57:27.297428 27889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:27.297674 27889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:27.297770 27889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:27.304823 27889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:27.304916 27889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:27.305020 27889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:27.313814 27889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:27.315040 27889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:27.316828 27889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:27.329465 27894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:27.329571 27895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:27.329504 27897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:27.330412 27889 server_base.cc:1034] running on GCE node
I20260928 15:57:27.331020 27889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:27.331866 27889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:27.333077 27889 hybrid_clock.cc:648] HybridClock initialized: now 1790611047333044 us; error 46 us; skew 500 ppm
I20260928 15:57:27.333407 27889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:27.335582 27889 webserver.cc:460] Webserver started at http://127.22.125.194:40933/ using document root <none> and password file <none>
I20260928 15:57:27.336323 27889 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:27.336437 27889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:27.336753 27889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:27.339293 27889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "24156158eedd46cfa293f36f6656772f"
format_stamp: "Formatted at 2026-09-28 15:57:27 on dist-test-slave-9cqd"
server_key: "c41eb436de9c00719d029d6acdde284b"
server_key_iv: "b7355724cd89449a2b260ad71aa50d0c"
server_key_version: "encryptionkey@0"
I20260928 15:57:27.340106 27889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "24156158eedd46cfa293f36f6656772f"
format_stamp: "Formatted at 2026-09-28 15:57:27 on dist-test-slave-9cqd"
server_key: "c41eb436de9c00719d029d6acdde284b"
server_key_iv: "b7355724cd89449a2b260ad71aa50d0c"
server_key_version: "encryptionkey@0"
I20260928 15:57:27.344934 27889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 15:57:27.348099 27903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:27.349329 27889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 15:57:27.349494 27889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "24156158eedd46cfa293f36f6656772f"
format_stamp: "Formatted at 2026-09-28 15:57:27 on dist-test-slave-9cqd"
server_key: "c41eb436de9c00719d029d6acdde284b"
server_key_iv: "b7355724cd89449a2b260ad71aa50d0c"
server_key_version: "encryptionkey@0"
I20260928 15:57:27.349702 27889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:27.355250 27889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:27.356000 27889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:27.356338 27889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:27.357172 27889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:27.358456 27889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:27.358577 27889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:27.358690 27889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:27.358777 27889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:27.390254 27889 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:35635
I20260928 15:57:27.390309 28015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:35635 every 8 connection(s)
I20260928 15:57:27.391544 27889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:27.394445 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 27889
I20260928 15:57:27.394634 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:27.395282 23031 external_mini_cluster.cc:1428] Setting key ee349e1cf4b62a5bb728b740e7f40261
I20260928 15:57:27.398052 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41875
--builtin_ntp_servers=127.22.125.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 15:57:27.400198 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.394009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:60387 (local address 127.22.125.254:41875)
0928 15:57:27.394220 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:27.394225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:27.396191 (+ 1966us) server_negotiation.cc:407] Connection header received
0928 15:57:27.397791 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:27.397796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:27.397884 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:27.398011 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:27.399425 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:27.399431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:27.399436 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:27.399460 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:27.399491 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:27.399501 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:27.399641 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:27.399827 (+ 186us) server_negotiation.cc:299] Negotiation successful
0928 15:57:27.399897 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260928 15:57:27.401096 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.394701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:41875 (local address 127.22.125.194:60387)
0928 15:57:27.395708 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:27.395753 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:27.397527 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:27.398253 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:27.398267 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:27.398385 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:27.399138 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:27.399141 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:27.399175 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:27.399256 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:27.399590 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:27.399600 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:27.399612 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 15:57:27.399906 (+ 294us) client_negotiation.cc:240] Negotiation successful
0928 15:57:27.400180 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":363,"threads_started":1}
I20260928 15:57:27.402377 28016 heartbeater.cc:347] Connected to a master server at 127.22.125.254:41875
I20260928 15:57:27.402673 28016 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:27.403267 28016 heartbeater.cc:511] Master 127.22.125.254:41875 requested a full tablet report, sending...
I20260928 15:57:27.404664 27698 ts_manager.cc:194] Registered new tserver with Master: 24156158eedd46cfa293f36f6656772f (127.22.125.194:35635)
W20260928 15:57:27.718663 28019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:27.718931 28019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:27.719038 28019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:27.726956 28019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:27.727059 28019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:27.727182 28019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:27.736868 28019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:27.738008 28019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:27.739709 28019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:27.752470 28027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:27.752532 28024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:27.752569 28025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:27.753355 28019 server_base.cc:1034] running on GCE node
I20260928 15:57:27.753975 28019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:27.754834 28019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:27.756105 28019 hybrid_clock.cc:648] HybridClock initialized: now 1790611047756007 us; error 52 us; skew 500 ppm
I20260928 15:57:27.756409 28019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:27.758538 28019 webserver.cc:460] Webserver started at http://127.22.125.195:33217/ using document root <none> and password file <none>
I20260928 15:57:27.759177 28019 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:27.759302 28019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:27.759624 28019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:27.762347 28019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "4a8b4e512ade450c997627b12c3da959"
format_stamp: "Formatted at 2026-09-28 15:57:27 on dist-test-slave-9cqd"
server_key: "1487c023cf993e91b4925d403c30c03a"
server_key_iv: "780f7f892571559e01232789420e9421"
server_key_version: "encryptionkey@0"
I20260928 15:57:27.763101 28019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "4a8b4e512ade450c997627b12c3da959"
format_stamp: "Formatted at 2026-09-28 15:57:27 on dist-test-slave-9cqd"
server_key: "1487c023cf993e91b4925d403c30c03a"
server_key_iv: "780f7f892571559e01232789420e9421"
server_key_version: "encryptionkey@0"
I20260928 15:57:27.767999 28019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 15:57:27.771127 28033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:27.772423 28019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:27.772578 28019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "4a8b4e512ade450c997627b12c3da959"
format_stamp: "Formatted at 2026-09-28 15:57:27 on dist-test-slave-9cqd"
server_key: "1487c023cf993e91b4925d403c30c03a"
server_key_iv: "780f7f892571559e01232789420e9421"
server_key_version: "encryptionkey@0"
I20260928 15:57:27.772781 28019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:27.778106 28019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:27.778834 28019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:27.779095 28019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:27.779938 28019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:27.781235 28019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:27.781365 28019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:27.781486 28019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:27.781569 28019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:27.813905 28019 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:46489
I20260928 15:57:27.813997 28147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:46489 every 8 connection(s)
I20260928 15:57:27.815253 28019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:27.816816 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28019
I20260928 15:57:27.817036 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:27.817580 23031 external_mini_cluster.cc:1428] Setting key 3eadea09e5b314bb9eb8776a161aea10
I20260928 15:57:27.823750 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.818128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:58597 (local address 127.22.125.254:41875)
0928 15:57:27.818329 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:27.818334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:27.820004 (+ 1670us) server_negotiation.cc:407] Connection header received
0928 15:57:27.821912 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:27.821916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:27.821965 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:27.822045 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:27.823191 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:27.823195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:27.823198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:27.823214 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:27.823244 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:27.823255 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:27.823342 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:27.823504 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 15:57:27.823551 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 15:57:27.824688 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.818725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:41875 (local address 127.22.125.195:58597)
0928 15:57:27.819848 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:27.819896 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:27.821664 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:27.822238 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:27.822246 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:27.822332 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:27.822926 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:27.822928 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:27.822964 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:27.823046 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:27.823333 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:27.823339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:27.823345 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 15:57:27.823552 (+ 207us) client_negotiation.cc:240] Negotiation successful
0928 15:57:27.823737 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":337,"threads_started":1}
I20260928 15:57:27.825793 28148 heartbeater.cc:347] Connected to a master server at 127.22.125.254:41875
I20260928 15:57:27.826066 28148 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:27.826613 28148 heartbeater.cc:511] Master 127.22.125.254:41875 requested a full tablet report, sending...
I20260928 15:57:27.827852 27698 ts_manager.cc:194] Registered new tserver with Master: 4a8b4e512ade450c997627b12c3da959 (127.22.125.195:46489)
I20260928 15:57:27.832883 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:57:27.839537 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.837282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.22.125.254:41875)
0928 15:57:27.837422 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:27.837427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:27.837992 (+ 565us) server_negotiation.cc:407] Connection header received
0928 15:57:27.838126 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:27.838129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:27.838178 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:27.838258 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:27.839076 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:27.839080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:27.839083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:27.839096 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:27.839124 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:27.839135 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:27.839225 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:27.839305 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 15:57:27.839352 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 15:57:27.843230 27698 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:27.849987 27698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:27.867954 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.862067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.22.125.194:35635)
0928 15:57:27.863769 (+ 1702us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:27.863776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:27.865329 (+ 1553us) server_negotiation.cc:407] Connection header received
0928 15:57:27.865587 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:27.865599 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:27.865832 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:27.866010 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:27.867078 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:27.867087 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:27.867103 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:27.867122 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:27.867178 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:27.867200 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:27.867385 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:27.867460 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 15:57:27.867559 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1554,"thread_start_us":1416,"threads_started":1}
I20260928 15:57:27.868085 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.861522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:35635 (local address 127.0.0.1:41916)
0928 15:57:27.864221 (+ 2699us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:27.864238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:27.865400 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:27.866007 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:27.866011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:27.866035 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:27.866700 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:27.866703 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:27.866718 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:27.866756 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:27.867309 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:27.867313 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:27.867318 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 15:57:27.867457 (+ 139us) client_negotiation.cc:240] Negotiation successful
0928 15:57:27.867555 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2568,"thread_start_us":2405,"threads_started":1}
I20260928 15:57:27.868170 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.861413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:33013 (local address 127.0.0.1:53368)
0928 15:57:27.863231 (+ 1818us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:27.863272 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:27.863425 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:27.865101 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:27.865108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:27.865198 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:27.866063 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:27.866069 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:27.866101 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:27.866209 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:27.867030 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:27.867048 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:27.867058 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 15:57:27.867530 (+ 472us) client_negotiation.cc:240] Negotiation successful
0928 15:57:27.867587 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"thread_start_us":1402,"threads_started":1}
I20260928 15:57:27.868558 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.861553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.22.125.193:33013)
0928 15:57:27.862792 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:27.862801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:27.863590 (+ 789us) server_negotiation.cc:407] Connection header received
0928 15:57:27.863668 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:27.863675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:27.863922 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:27.864108 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:27.866700 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:27.866709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:27.866722 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:27.866749 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:27.866807 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:27.866828 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:27.866967 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:27.867578 (+ 611us) server_negotiation.cc:299] Negotiation successful
0928 15:57:27.867670 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064,"thread_start_us":954,"threads_started":1}
I20260928 15:57:27.869632 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.863736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:46489 (local address 127.0.0.1:51938)
0928 15:57:27.864735 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:27.864757 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:27.864887 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:27.866570 (+ 1683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:27.866576 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:27.866603 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:27.868588 (+ 1985us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:27.868593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:27.868610 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:27.868647 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:27.869066 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:27.869071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:27.869076 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 15:57:27.869200 (+ 124us) client_negotiation.cc:240] Negotiation successful
0928 15:57:27.869245 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":873,"threads_started":1}
I20260928 15:57:27.869668 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:27.864338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.22.125.195:46489)
0928 15:57:27.866041 (+ 1703us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:27.866049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:27.866082 (+ 33us) server_negotiation.cc:407] Connection header received
0928 15:57:27.866161 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:27.866169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:27.866365 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:27.866511 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:27.868819 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:27.868830 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:27.868844 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:27.868865 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:27.868923 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:27.868946 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:27.869121 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:27.869191 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 15:57:27.869276 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1523,"thread_start_us":1331,"threads_started":1}
I20260928 15:57:27.872186 27819 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9e8224dd494335acd54aac88657af6 (DEFAULT_TABLE table=test-table [id=f824d4a720c749adb720776aae75f2d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:27.872167 27951 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9e8224dd494335acd54aac88657af6 (DEFAULT_TABLE table=test-table [id=f824d4a720c749adb720776aae75f2d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:27.873525 27951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9e8224dd494335acd54aac88657af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:27.873075 28082 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9e8224dd494335acd54aac88657af6 (DEFAULT_TABLE table=test-table [id=f824d4a720c749adb720776aae75f2d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:27.873683 27819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9e8224dd494335acd54aac88657af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:27.874295 28082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9e8224dd494335acd54aac88657af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:27.884792 28166 tablet_bootstrap.cc:492] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Bootstrap starting.
I20260928 15:57:27.887908 28167 tablet_bootstrap.cc:492] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Bootstrap starting.
I20260928 15:57:27.888388 28166 tablet_bootstrap.cc:654] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:27.888772 28168 tablet_bootstrap.cc:492] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Bootstrap starting.
I20260928 15:57:27.889719 28166 log.cc:824] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:27.892148 28167 tablet_bootstrap.cc:654] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:27.892300 28168 tablet_bootstrap.cc:654] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:27.893098 28167 log.cc:824] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:27.893472 28168 log.cc:824] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:27.893591 28166 tablet_bootstrap.cc:492] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: No bootstrap required, opened a new log
I20260928 15:57:27.893877 28166 ts_tablet_manager.cc:1397] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260928 15:57:27.896010 28168 tablet_bootstrap.cc:492] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: No bootstrap required, opened a new log
I20260928 15:57:27.896100 28167 tablet_bootstrap.cc:492] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: No bootstrap required, opened a new log
I20260928 15:57:27.896291 28168 ts_tablet_manager.cc:1397] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260928 15:57:27.896387 28167 ts_tablet_manager.cc:1397] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260928 15:57:27.903584 28168 raft_consensus.cc:348] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:27.903934 28168 raft_consensus.cc:374] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:27.904081 28168 raft_consensus.cc:729] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a8b4e512ade450c997627b12c3da959, State: Initialized, Role: FOLLOWER
I20260928 15:57:27.904677 28168 consensus_queue.cc:260] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:27.904561 28166 raft_consensus.cc:348] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:27.904951 28166 raft_consensus.cc:374] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:27.905076 28166 raft_consensus.cc:729] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24156158eedd46cfa293f36f6656772f, State: Initialized, Role: FOLLOWER
I20260928 15:57:27.905658 28166 consensus_queue.cc:260] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:27.906940 28167 raft_consensus.cc:348] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:27.907428 28167 raft_consensus.cc:374] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:27.907598 28167 raft_consensus.cc:729] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e13fe0aa804e969118a93e35555211, State: Initialized, Role: FOLLOWER
I20260928 15:57:27.908152 28016 heartbeater.cc:503] Master 127.22.125.254:41875 was elected leader, sending a full tablet report...
I20260928 15:57:27.908327 28167 consensus_queue.cc:260] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:27.908704 28148 heartbeater.cc:503] Master 127.22.125.254:41875 was elected leader, sending a full tablet report...
I20260928 15:57:27.909305 28168 ts_tablet_manager.cc:1428] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260928 15:57:27.910506 28166 ts_tablet_manager.cc:1428] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Time spent starting tablet: real 0.016s user 0.005s sys 0.009s
I20260928 15:57:27.911453 27885 heartbeater.cc:503] Master 127.22.125.254:41875 was elected leader, sending a full tablet report...
I20260928 15:57:27.912595 28167 ts_tablet_manager.cc:1428] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260928 15:57:27.982079 27886 tablet.cc:2369] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:57:28.066988 28149 tablet.cc:2369] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:57:28.087433 28173 raft_consensus.cc:482] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:28.087759 28173 raft_consensus.cc:504] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:28.089341 28173 leader_election.cc:288] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9e13fe0aa804e969118a93e35555211 (127.22.125.193:33013), 4a8b4e512ade450c997627b12c3da959 (127.22.125.195:46489)
I20260928 15:57:28.092523 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.089826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:32827 (local address 127.22.125.193:33013)
0928 15:57:28.089999 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:28.090004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:28.090471 (+ 467us) server_negotiation.cc:407] Connection header received
0928 15:57:28.090615 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:28.090623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:28.090692 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:28.090814 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:28.091851 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:28.091855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:28.091860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:28.091876 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:28.091906 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:28.091916 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:28.092033 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:28.092179 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 15:57:28.092238 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 15:57:28.092659 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.089648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:33013 (local address 127.22.125.194:32827)
0928 15:57:28.090322 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:28.090342 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:28.090469 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:28.090852 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:28.090867 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:28.090898 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:28.091651 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:28.091653 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:28.091670 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:28.091709 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:28.092006 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:28.092011 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:28.092015 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 15:57:28.092179 (+ 164us) client_negotiation.cc:240] Negotiation successful
0928 15:57:28.092233 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":342,"threads_started":1}
I20260928 15:57:28.092957 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.090550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:39863 (local address 127.22.125.195:46489)
0928 15:57:28.090735 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:28.090743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:28.091173 (+ 430us) server_negotiation.cc:407] Connection header received
0928 15:57:28.091251 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:28.091254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:28.091301 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:28.091379 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:28.092360 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:28.092366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:28.092371 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:28.092389 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:28.092425 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:28.092436 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:28.092562 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:28.092668 (+ 106us) server_negotiation.cc:299] Negotiation successful
0928 15:57:28.092729 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 15:57:28.093515 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.090408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:46489 (local address 127.22.125.194:39863)
0928 15:57:28.091065 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:28.091079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:28.091162 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:28.091437 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:28.091442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:28.091467 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:28.092162 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:57:28.092164 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:28.092176 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:57:28.092210 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:28.092525 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:28.092529 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:28.092534 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 15:57:28.093197 (+ 663us) client_negotiation.cc:240] Negotiation successful
0928 15:57:28.093246 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":349,"threads_started":1}
I20260928 15:57:28.093737 27839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e8224dd494335acd54aac88657af6" candidate_uuid: "24156158eedd46cfa293f36f6656772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e13fe0aa804e969118a93e35555211" is_pre_election: true
I20260928 15:57:28.094348 27839 raft_consensus.cc:2454] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24156158eedd46cfa293f36f6656772f in term 0.
I20260928 15:57:28.094564 28102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e8224dd494335acd54aac88657af6" candidate_uuid: "24156158eedd46cfa293f36f6656772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a8b4e512ade450c997627b12c3da959" is_pre_election: true
I20260928 15:57:28.095006 27905 leader_election.cc:302] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24156158eedd46cfa293f36f6656772f, f9e13fe0aa804e969118a93e35555211; no voters:
I20260928 15:57:28.095067 28102 raft_consensus.cc:2454] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24156158eedd46cfa293f36f6656772f in term 0.
I20260928 15:57:28.095419 28173 raft_consensus.cc:2781] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:28.095579 28173 raft_consensus.cc:482] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:28.095695 28173 raft_consensus.cc:3037] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:28.098033 28173 raft_consensus.cc:504] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:28.098632 28173 leader_election.cc:288] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [CANDIDATE]: Term 1 election: Requested vote from peers f9e13fe0aa804e969118a93e35555211 (127.22.125.193:33013), 4a8b4e512ade450c997627b12c3da959 (127.22.125.195:46489)
I20260928 15:57:28.099081 27839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e8224dd494335acd54aac88657af6" candidate_uuid: "24156158eedd46cfa293f36f6656772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e13fe0aa804e969118a93e35555211"
I20260928 15:57:28.099254 28102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9e8224dd494335acd54aac88657af6" candidate_uuid: "24156158eedd46cfa293f36f6656772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a8b4e512ade450c997627b12c3da959"
I20260928 15:57:28.099308 27839 raft_consensus.cc:3037] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:28.099457 28102 raft_consensus.cc:3037] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:28.101568 27839 raft_consensus.cc:2454] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24156158eedd46cfa293f36f6656772f in term 1.
I20260928 15:57:28.101706 28102 raft_consensus.cc:2454] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24156158eedd46cfa293f36f6656772f in term 1.
I20260928 15:57:28.102007 27905 leader_election.cc:302] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24156158eedd46cfa293f36f6656772f, f9e13fe0aa804e969118a93e35555211; no voters:
I20260928 15:57:28.102335 28173 raft_consensus.cc:2781] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:28.102746 28173 raft_consensus.cc:686] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 1 LEADER]: Becoming Leader. State: Replica: 24156158eedd46cfa293f36f6656772f, State: Running, Role: LEADER
I20260928 15:57:28.103171 28173 consensus_queue.cc:237] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } }
I20260928 15:57:28.107511 27696 catalog_manager.cc:5495] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f reported cstate change: term changed from 0 to 1, leader changed from <none> to 24156158eedd46cfa293f36f6656772f (127.22.125.194). New cstate: current_term: 1 leader_uuid: "24156158eedd46cfa293f36f6656772f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24156158eedd46cfa293f36f6656772f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 } health_report { overall_health: UNKNOWN } } }
W20260928 15:57:28.143687 28017 tablet.cc:2369] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:57:28.162317 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.160091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.22.125.194:35635)
0928 15:57:28.160286 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:28.160292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:28.160405 (+ 113us) server_negotiation.cc:407] Connection header received
0928 15:57:28.160654 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:28.160658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:28.160736 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:28.160845 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:28.161720 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:28.161726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:28.161731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:28.161748 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:28.161786 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:28.161797 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:28.161921 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:28.162014 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 15:57:28.162074 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 15:57:28.170288 27839 raft_consensus.cc:1261] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 1 FOLLOWER]: Refusing update from remote peer 24156158eedd46cfa293f36f6656772f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:28.170310 28102 raft_consensus.cc:1261] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 1 FOLLOWER]: Refusing update from remote peer 24156158eedd46cfa293f36f6656772f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:28.171190 28177 consensus_queue.cc:1035] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a8b4e512ade450c997627b12c3da959" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:28.171546 28173 consensus_queue.cc:1035] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e13fe0aa804e969118a93e35555211" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 33013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:28.183638 28181 mvcc.cc:205] Tried to move back new op lower bound from 7334342853295624192 to 7334342853034360832. Current Snapshot: MvccSnapshot[applied={T|T < 7334342853295624192}]
I20260928 15:57:28.186689 28180 mvcc.cc:205] Tried to move back new op lower bound from 7334342853295624192 to 7334342853034360832. Current Snapshot: MvccSnapshot[applied={T|T < 7334342853295624192 or (T in {7334342853295624192})}]
I20260928 15:57:28.197062 27696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:28.197314 27696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:28.200796 27696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 042246b9cbdf4174984261a9fe87bf2a: Sending DeleteTablet for 3 replicas of tablet 9b9e8224dd494335acd54aac88657af6
I20260928 15:57:28.201948 27819 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9e8224dd494335acd54aac88657af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:28 UTC) from {username='slave'} at 127.0.0.1:53368
I20260928 15:57:28.201996 27951 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9e8224dd494335acd54aac88657af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:28 UTC) from {username='slave'} at 127.0.0.1:41916
I20260928 15:57:28.202781 28190 tablet_replica.cc:330] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: stopping tablet replica
I20260928 15:57:28.202847 28082 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9e8224dd494335acd54aac88657af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:28 UTC) from {username='slave'} at 127.0.0.1:51938
I20260928 15:57:28.203342 28190 raft_consensus.cc:2229] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:28.205533 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 27757
I20260928 15:57:28.213492 28192 tablet_replica.cc:330] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: stopping tablet replica
I20260928 15:57:28.214008 28192 raft_consensus.cc:2229] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:28.214416 28190 raft_consensus.cc:2258] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:28.214579 28192 raft_consensus.cc:2258] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:28.216943 28192 ts_tablet_manager.cc:1905] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:28.216943 28190 ts_tablet_manager.cc:1905] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:28.224185 28192 ts_tablet_manager.cc:1918] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:28.224386 28192 log.cc:1196] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/9b9e8224dd494335acd54aac88657af6
I20260928 15:57:28.225030 28192 ts_tablet_manager.cc:1939] T 9b9e8224dd494335acd54aac88657af6 P 4a8b4e512ade450c997627b12c3da959: Deleting consensus metadata
I20260928 15:57:28.227320 27683 catalog_manager.cc:4841] TS 4a8b4e512ade450c997627b12c3da959 (127.22.125.195:46489): tablet 9b9e8224dd494335acd54aac88657af6 (table test-table [id=f824d4a720c749adb720776aae75f2d3]) successfully deleted
I20260928 15:57:28.228185 28191 tablet_replica.cc:330] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: stopping tablet replica
I20260928 15:57:28.228680 28191 raft_consensus.cc:2229] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:28.229334 28191 raft_consensus.cc:2258] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:28.230400 28190 ts_tablet_manager.cc:1918] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:28.230561 28190 log.cc:1196] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/9b9e8224dd494335acd54aac88657af6
I20260928 15:57:28.230887 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.207216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.22.125.193:33013)
0928 15:57:28.207431 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:28.207440 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:28.227903 (+ 20463us) server_negotiation.cc:407] Connection header received
0928 15:57:28.228237 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:28.228244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:28.228369 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:28.228447 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:28.229907 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:28.229915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:28.229921 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:28.229943 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:28.229976 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:28.229992 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:28.230097 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:28.230369 (+ 272us) server_negotiation.cc:299] Negotiation successful
0928 15:57:28.230439 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 15:57:28.231114 28190 ts_tablet_manager.cc:1939] T 9b9e8224dd494335acd54aac88657af6 P f9e13fe0aa804e969118a93e35555211: Deleting consensus metadata
I20260928 15:57:28.231698 28191 ts_tablet_manager.cc:1905] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:28.232132 27880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:28.239320 28191 ts_tablet_manager.cc:1918] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:28.239509 28191 log.cc:1196] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/9b9e8224dd494335acd54aac88657af6
I20260928 15:57:28.240099 28191 ts_tablet_manager.cc:1939] T 9b9e8224dd494335acd54aac88657af6 P 24156158eedd46cfa293f36f6656772f: Deleting consensus metadata
I20260928 15:57:28.241833 27682 catalog_manager.cc:4841] TS 24156158eedd46cfa293f36f6656772f (127.22.125.194:35635): tablet 9b9e8224dd494335acd54aac88657af6 (table test-table [id=f824d4a720c749adb720776aae75f2d3]) successfully deleted
I20260928 15:57:28.322176 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 27757
I20260928 15:57:28.322319 27683 catalog_manager.cc:4841] TS f9e13fe0aa804e969118a93e35555211 (127.22.125.193:33013): tablet 9b9e8224dd494335acd54aac88657af6 (table test-table [id=f824d4a720c749adb720776aae75f2d3]) successfully deleted
I20260928 15:57:28.338654 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 27889
I20260928 15:57:28.341814 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.339532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.22.125.194:35635)
0928 15:57:28.339939 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:28.339945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:28.339965 (+ 20us) server_negotiation.cc:407] Connection header received
0928 15:57:28.340078 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:28.340084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:28.340148 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:28.340250 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:28.341142 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:28.341148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:28.341152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:28.341170 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:28.341208 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:28.341224 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:28.341370 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:28.341431 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 15:57:28.341488 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260928 15:57:28.342612 28011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:28.450974 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 27889
I20260928 15:57:28.469188 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28019
I20260928 15:57:28.471818 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:28.469990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.22.125.195:46489)
0928 15:57:28.470153 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:28.470157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:28.470175 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:57:28.470313 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:28.470316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:28.470410 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:28.470497 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:28.471345 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:28.471348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:28.471351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:28.471370 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:28.471395 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:28.471406 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:28.471505 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:28.471565 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 15:57:28.471607 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 15:57:28.472501 28143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:28.563630 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28019
I20260928 15:57:28.579411 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 27667
I20260928 15:57:28.580575 27728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:28.692837 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 27667
2026-09-28T15:57:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2694 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 15:57:28.733438 23031 mini_oidc.cc:152] Starting JWKS server
I20260928 15:57:28.737533 23031 webserver.cc:460] Webserver started at https://127.0.0.1:33171/ using document root <none> and password file <none>
I20260928 15:57:28.737849 23031 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 15:57:28.738497 23031 webserver.cc:460] Webserver started at http://127.0.0.1:46403/ using document root <none> and password file <none>
2026-09-28T15:57:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:28Z Disabled control of system clock
I20260928 15:57:28.763803 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:43981
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:45999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:43981
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33171/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/testchainca.cert with env {}
W20260928 15:57:29.078750 28203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:29.079032 28203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:29.079119 28203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:29.086162 28203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:29.086274 28203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:29.086313 28203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:29.086369 28203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:29.086417 28203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 15:57:29.086472 28203 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 15:57:29.086510 28203 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33171/jwks.json
I20260928 15:57:29.096436 28203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:43981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:43981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33171/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:29.097621 28203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:29.098927 28203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:29.110455 28209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:29.110497 28212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:29.110529 28210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:29.111382 28203 server_base.cc:1034] running on GCE node
I20260928 15:57:29.112164 28203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:29.113478 28203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:29.114710 28203 hybrid_clock.cc:648] HybridClock initialized: now 1790611049114668 us; error 48 us; skew 500 ppm
I20260928 15:57:29.115056 28203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:29.117331 28203 webserver.cc:460] Webserver started at http://127.22.125.254:39745/ using document root <none> and password file <none>
I20260928 15:57:29.118008 28203 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:29.118137 28203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:29.118453 28203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:29.121402 28203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "bc83465e4505409c8b046baae67039aa"
format_stamp: "Formatted at 2026-09-28 15:57:29 on dist-test-slave-9cqd"
server_key: "3072fac67de265b99b10de75d71264f8"
server_key_iv: "89400ac68c3394d0c4508e96f28cd6fd"
server_key_version: "encryptionkey@0"
I20260928 15:57:29.122174 28203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "bc83465e4505409c8b046baae67039aa"
format_stamp: "Formatted at 2026-09-28 15:57:29 on dist-test-slave-9cqd"
server_key: "3072fac67de265b99b10de75d71264f8"
server_key_iv: "89400ac68c3394d0c4508e96f28cd6fd"
server_key_version: "encryptionkey@0"
I20260928 15:57:29.127086 28203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 15:57:29.130445 28218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:29.131685 28203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 15:57:29.131858 28203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "bc83465e4505409c8b046baae67039aa"
format_stamp: "Formatted at 2026-09-28 15:57:29 on dist-test-slave-9cqd"
server_key: "3072fac67de265b99b10de75d71264f8"
server_key_iv: "89400ac68c3394d0c4508e96f28cd6fd"
server_key_version: "encryptionkey@0"
I20260928 15:57:29.132091 28203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:29.146027 28203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:29.146791 28203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:29.147080 28203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:29.164609 28203 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:43981
I20260928 15:57:29.164642 28269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:43981 every 8 connection(s)
I20260928 15:57:29.165887 28203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:29.169456 28270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:29.169988 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28203
I20260928 15:57:29.170207 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:29.170842 23031 external_mini_cluster.cc:1428] Setting key 1a58d0ec57c84f93b13af45ffd384ed2
I20260928 15:57:29.183666 28270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa: Bootstrap starting.
I20260928 15:57:29.185576 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:29.172783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.22.125.254:43981)
0928 15:57:29.173856 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:29.173875 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:29.173931 (+ 56us) server_negotiation.cc:407] Connection header received
0928 15:57:29.175929 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:29.175954 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:29.176548 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:29.177005 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:29.177514 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.179352 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.180703 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.181727 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.181909 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:29.182973 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:29.182984 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:29.182999 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:29.183663 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:29.183760 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:29.183785 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:29.183902 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:29.184081 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 15:57:29.184375 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":335,"threads_started":1}
I20260928 15:57:29.187366 28270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:29.188791 28270 log.cc:824] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:29.189597 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 15:57:29.191440 28270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa: No bootstrap required, opened a new log
I20260928 15:57:29.199080 28270 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc83465e4505409c8b046baae67039aa" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43981 } }
I20260928 15:57:29.199426 28270 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:29.199541 28270 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc83465e4505409c8b046baae67039aa, State: Initialized, Role: FOLLOWER
I20260928 15:57:29.200119 28270 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc83465e4505409c8b046baae67039aa" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43981 } }
I20260928 15:57:29.200368 28270 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:29.200481 28270 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:29.200634 28270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:29.203130 28270 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc83465e4505409c8b046baae67039aa" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43981 } }
I20260928 15:57:29.203611 28270 leader_election.cc:302] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc83465e4505409c8b046baae67039aa; no voters:
I20260928 15:57:29.204217 28270 leader_election.cc:288] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:29.204370 28275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:29.205101 28275 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [term 1 LEADER]: Becoming Leader. State: Replica: bc83465e4505409c8b046baae67039aa, State: Running, Role: LEADER
I20260928 15:57:29.205677 28275 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc83465e4505409c8b046baae67039aa" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43981 } }
I20260928 15:57:29.206234 28270 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:29.208482 28276 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc83465e4505409c8b046baae67039aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc83465e4505409c8b046baae67039aa" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43981 } } }
I20260928 15:57:29.208842 28276 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:29.211187 28282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:29.211018 28277 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc83465e4505409c8b046baae67039aa. Latest consensus state: current_term: 1 leader_uuid: "bc83465e4505409c8b046baae67039aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc83465e4505409c8b046baae67039aa" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43981 } } }
I20260928 15:57:29.211328 28277 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:29.216426 28282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:29.224954 28282 catalog_manager.cc:1347] Generated new cluster ID: 541b99c32ec64aefb690bc63a3e2ef40
I20260928 15:57:29.225059 28282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:29.253657 28282 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:29.254894 28282 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:29.260943 28282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc83465e4505409c8b046baae67039aa: Generated new TSK 0
I20260928 15:57:29.261698 28282 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 15:57:29.316702 28300 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 15:57:29.316880 28300 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 15:57:29.318379 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:29.288215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.22.125.254:43981)
0928 15:57:29.288451 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:29.288458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:29.288889 (+ 431us) server_negotiation.cc:407] Connection header received
0928 15:57:29.289295 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:29.289299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:29.289369 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:29.289488 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0928 15:57:29.290117 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.290854 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.292998 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.293458 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.293551 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:29.294150 (+ 599us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 15:57:29.317833 (+ 23683us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33171/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 15:57:29.318147 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33171/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":89}
W20260928 15:57:29.318526 28273 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33171/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260928 15:57:29.320647 28299 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:29.288014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:43981 (local address 127.0.0.1:58284)
0928 15:57:29.288753 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:29.288775 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:29.288998 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:29.289548 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:29.289554 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:29.289567 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0928 15:57:29.289977 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:29.289991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.290995 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.291002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:29.292853 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:29.292880 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.293632 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.293639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:29.293826 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:29.293908 (+ 82us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 15:57:29.318164 (+ 24256us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33171/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 15:57:29.318529 (+ 365us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.125.254:43981: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33171/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":341,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.125.254:43981: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33171/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 15:57:29.322360 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28203
I20260928 15:57:29.323508 28265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:29.418984 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28203
2026-09-28T15:57:29Z chronyd exiting
I20260928 15:57:29.469312 23031 test_util.cc:175] -----------------------------------------------
I20260928 15:57:29.469436 23031 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790611005938867-23031-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (740 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 15:57:29.473027 23031 mini_oidc.cc:152] Starting JWKS server
I20260928 15:57:29.474828 23031 webserver.cc:460] Webserver started at https://127.0.0.1:33909/ using document root <none> and password file <none>
I20260928 15:57:29.475045 23031 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 15:57:29.475610 23031 webserver.cc:460] Webserver started at http://127.0.0.1:34829/ using document root <none> and password file <none>
2026-09-28T15:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:29Z Disabled control of system clock
I20260928 15:57:29.501392 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39249
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:46557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:39249
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33909/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/testchainca.cert with env {}
W20260928 15:57:29.816718 28307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:29.816998 28307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:29.817085 28307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:29.824651 28307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:29.824759 28307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:29.824797 28307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:29.824846 28307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:29.824883 28307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 15:57:29.824944 28307 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 15:57:29.824985 28307 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33909/jwks.json
I20260928 15:57:29.834877 28307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:46557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:39249
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33909/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:29.836087 28307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:29.837325 28307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:29.849020 28312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:29.849156 28313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:29.849162 28315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:29.849741 28307 server_base.cc:1034] running on GCE node
I20260928 15:57:29.850405 28307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:29.851701 28307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:29.852922 28307 hybrid_clock.cc:648] HybridClock initialized: now 1790611049852879 us; error 47 us; skew 500 ppm
I20260928 15:57:29.853269 28307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:29.855383 28307 webserver.cc:460] Webserver started at http://127.22.125.254:39827/ using document root <none> and password file <none>
I20260928 15:57:29.856155 28307 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:29.856285 28307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:29.856606 28307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:29.859356 28307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "40054331298e42a5922085aea3c088d5"
format_stamp: "Formatted at 2026-09-28 15:57:29 on dist-test-slave-9cqd"
server_key: "586e55890178218dc4b3c57e757139f8"
server_key_iv: "e301f2a5669225e3f7556108aee8d21c"
server_key_version: "encryptionkey@0"
I20260928 15:57:29.860185 28307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "40054331298e42a5922085aea3c088d5"
format_stamp: "Formatted at 2026-09-28 15:57:29 on dist-test-slave-9cqd"
server_key: "586e55890178218dc4b3c57e757139f8"
server_key_iv: "e301f2a5669225e3f7556108aee8d21c"
server_key_version: "encryptionkey@0"
I20260928 15:57:29.865125 28307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 15:57:29.868444 28321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:29.869696 28307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 15:57:29.869887 28307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "40054331298e42a5922085aea3c088d5"
format_stamp: "Formatted at 2026-09-28 15:57:29 on dist-test-slave-9cqd"
server_key: "586e55890178218dc4b3c57e757139f8"
server_key_iv: "e301f2a5669225e3f7556108aee8d21c"
server_key_version: "encryptionkey@0"
I20260928 15:57:29.870090 28307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:29.896055 28307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:29.896868 28307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:29.897171 28307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:29.914995 28307 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:39249
I20260928 15:57:29.915014 28372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:39249 every 8 connection(s)
I20260928 15:57:29.916402 28307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:29.916894 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28307
I20260928 15:57:29.917114 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:29.917919 23031 external_mini_cluster.cc:1428] Setting key 72447fa32b520ba7ee99ef545f5b13d2
I20260928 15:57:29.920791 28373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:29.932847 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:29.919770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.22.125.254:39249)
0928 15:57:29.920831 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:29.920847 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:29.920899 (+ 52us) server_negotiation.cc:407] Connection header received
0928 15:57:29.923085 (+ 2186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:29.923113 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:29.923695 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:29.924131 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:29.924630 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.926673 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.927864 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:29.928874 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:29.929062 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:29.930095 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:29.930109 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:29.930124 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:29.930877 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:29.930968 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:29.930997 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:29.931111 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:29.931306 (+ 195us) server_negotiation.cc:299] Negotiation successful
0928 15:57:29.931635 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":329,"threads_started":1}
I20260928 15:57:29.934248 28373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5: Bootstrap starting.
I20260928 15:57:29.936302 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 15:57:29.937417 28373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:29.938418 28373 log.cc:824] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:29.941179 28373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5: No bootstrap required, opened a new log
I20260928 15:57:29.948642 28373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40054331298e42a5922085aea3c088d5" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39249 } }
I20260928 15:57:29.948964 28373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:29.949067 28373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40054331298e42a5922085aea3c088d5, State: Initialized, Role: FOLLOWER
I20260928 15:57:29.949555 28373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40054331298e42a5922085aea3c088d5" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39249 } }
I20260928 15:57:29.949788 28373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:29.949920 28373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:29.950042 28373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:29.952255 28373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40054331298e42a5922085aea3c088d5" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39249 } }
I20260928 15:57:29.952661 28373 leader_election.cc:302] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40054331298e42a5922085aea3c088d5; no voters:
I20260928 15:57:29.953094 28373 leader_election.cc:288] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:29.953236 28378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:29.953904 28378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [term 1 LEADER]: Becoming Leader. State: Replica: 40054331298e42a5922085aea3c088d5, State: Running, Role: LEADER
I20260928 15:57:29.954429 28378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40054331298e42a5922085aea3c088d5" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39249 } }
I20260928 15:57:29.954921 28373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:29.957238 28380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40054331298e42a5922085aea3c088d5. Latest consensus state: current_term: 1 leader_uuid: "40054331298e42a5922085aea3c088d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40054331298e42a5922085aea3c088d5" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39249 } } }
I20260928 15:57:29.957257 28379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40054331298e42a5922085aea3c088d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40054331298e42a5922085aea3c088d5" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39249 } } }
I20260928 15:57:29.957729 28380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:29.957726 28379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:29.960181 28384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:29.965871 28384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:29.975564 28384 catalog_manager.cc:1347] Generated new cluster ID: 10d787923e9746579716baecd32083df
I20260928 15:57:29.975672 28384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:29.987730 28384 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:29.989094 28384 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:29.999675 28384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40054331298e42a5922085aea3c088d5: Generated new TSK 0
I20260928 15:57:30.000519 28384 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 15:57:30.042464 28404 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 15:57:30.042590 28404 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 15:57:30.043946 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:30.024071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.22.125.254:39249)
0928 15:57:30.024255 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:30.024262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:30.024699 (+ 437us) server_negotiation.cc:407] Connection header received
0928 15:57:30.024900 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:30.024906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:30.024999 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:30.025152 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0928 15:57:30.025586 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.026745 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.028069 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.028795 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.028918 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:30.029389 (+ 471us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 15:57:30.043199 (+ 13810us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 15:57:30.043626 (+ 427us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38964: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260928 15:57:30.044042 28403 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:30.023926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39249 (local address 127.0.0.1:38964)
0928 15:57:30.024607 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:30.024622 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:30.024734 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:30.025168 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:30.025171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:30.025181 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0928 15:57:30.025428 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:30.025436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.026883 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.026887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:30.027921 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:30.027931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.028951 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.028955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:30.029099 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:30.029130 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 15:57:30.043475 (+ 14345us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 15:57:30.043718 (+ 243us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.125.254:39249: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":407,"threads_started":1}
W20260928 15:57:30.044103 28376 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38964: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260928 15:57:30.045791 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28307
I20260928 15:57:30.046865 28368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:30.140326 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28307
2026-09-28T15:57:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (721 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 15:57:30.194934 23031 mini_oidc.cc:152] Starting JWKS server
I20260928 15:57:30.196904 23031 webserver.cc:460] Webserver started at https://127.0.0.1:40727/ using document root <none> and password file <none>
I20260928 15:57:30.197141 23031 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 15:57:30.197753 23031 webserver.cc:460] Webserver started at http://127.0.0.1:38455/ using document root <none> and password file <none>
2026-09-28T15:57:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:30Z Disabled control of system clock
I20260928 15:57:30.222759 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39567
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:39567
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40727/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 15:57:30.538161 28411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:30.538501 28411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:30.538653 28411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:30.545899 28411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:30.546038 28411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:30.546133 28411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:30.546231 28411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:30.546308 28411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 15:57:30.546396 28411 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 15:57:30.546466 28411 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40727/jwks.json
I20260928 15:57:30.555539 28411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:39567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40727/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:30.556856 28411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:30.558619 28411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:30.570482 28416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:30.570448 28419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:30.570513 28417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:30.571138 28411 server_base.cc:1034] running on GCE node
I20260928 15:57:30.571835 28411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:30.573205 28411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:30.574434 28411 hybrid_clock.cc:648] HybridClock initialized: now 1790611050574403 us; error 44 us; skew 500 ppm
I20260928 15:57:30.574774 28411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:30.576973 28411 webserver.cc:460] Webserver started at http://127.22.125.254:42979/ using document root <none> and password file <none>
I20260928 15:57:30.577652 28411 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:30.577785 28411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:30.578161 28411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:30.581017 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "ca67a67dc163402b91f45d99906b9cd8"
format_stamp: "Formatted at 2026-09-28 15:57:30 on dist-test-slave-9cqd"
server_key: "931c87e1bfb34737f6a8b0744772a6dc"
server_key_iv: "526c9f5317ebbe28fb0b2d96c8a9ee1b"
server_key_version: "encryptionkey@0"
I20260928 15:57:30.581794 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "ca67a67dc163402b91f45d99906b9cd8"
format_stamp: "Formatted at 2026-09-28 15:57:30 on dist-test-slave-9cqd"
server_key: "931c87e1bfb34737f6a8b0744772a6dc"
server_key_iv: "526c9f5317ebbe28fb0b2d96c8a9ee1b"
server_key_version: "encryptionkey@0"
I20260928 15:57:30.586679 28411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 15:57:30.590019 28425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:30.591343 28411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 15:57:30.591545 28411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "ca67a67dc163402b91f45d99906b9cd8"
format_stamp: "Formatted at 2026-09-28 15:57:30 on dist-test-slave-9cqd"
server_key: "931c87e1bfb34737f6a8b0744772a6dc"
server_key_iv: "526c9f5317ebbe28fb0b2d96c8a9ee1b"
server_key_version: "encryptionkey@0"
I20260928 15:57:30.591790 28411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:30.611440 28411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:30.612386 28411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:30.612680 28411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:30.630330 28411 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:39567
I20260928 15:57:30.630358 28477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:39567 every 8 connection(s)
I20260928 15:57:30.631675 28411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:30.633248 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28411
I20260928 15:57:30.633479 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:30.634289 23031 external_mini_cluster.cc:1428] Setting key b936adcb95996d1ddc829a5e6d588cf6
I20260928 15:57:30.636384 28478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:30.647552 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:30.636203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.22.125.254:39567)
0928 15:57:30.637169 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:30.637180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:30.637218 (+ 38us) server_negotiation.cc:407] Connection header received
0928 15:57:30.638744 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:30.638764 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:30.639218 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:30.639531 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:30.640160 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.641464 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.643360 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.643972 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.644176 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:30.645355 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:30.645363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:30.645375 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:30.645927 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:30.646002 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:30.646029 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:30.646112 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:30.646357 (+ 245us) server_negotiation.cc:299] Negotiation successful
0928 15:57:30.646680 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":368,"threads_started":1}
I20260928 15:57:30.648808 28478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8: Bootstrap starting.
I20260928 15:57:30.651232 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 15:57:30.651984 28478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:30.652920 28478 log.cc:824] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:30.655630 28478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8: No bootstrap required, opened a new log
I20260928 15:57:30.662928 28478 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67a67dc163402b91f45d99906b9cd8" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39567 } }
I20260928 15:57:30.663383 28478 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:30.663506 28478 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca67a67dc163402b91f45d99906b9cd8, State: Initialized, Role: FOLLOWER
I20260928 15:57:30.664157 28478 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67a67dc163402b91f45d99906b9cd8" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39567 } }
I20260928 15:57:30.664367 28478 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:30.664462 28478 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:30.664599 28478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:30.666600 28478 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67a67dc163402b91f45d99906b9cd8" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39567 } }
I20260928 15:57:30.667019 28478 leader_election.cc:302] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca67a67dc163402b91f45d99906b9cd8; no voters:
I20260928 15:57:30.667717 28478 leader_election.cc:288] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:30.667829 28483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:30.668672 28483 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [term 1 LEADER]: Becoming Leader. State: Replica: ca67a67dc163402b91f45d99906b9cd8, State: Running, Role: LEADER
I20260928 15:57:30.669224 28483 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67a67dc163402b91f45d99906b9cd8" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39567 } }
I20260928 15:57:30.669751 28478 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:30.672358 28484 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca67a67dc163402b91f45d99906b9cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67a67dc163402b91f45d99906b9cd8" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39567 } } }
I20260928 15:57:30.672389 28485 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca67a67dc163402b91f45d99906b9cd8. Latest consensus state: current_term: 1 leader_uuid: "ca67a67dc163402b91f45d99906b9cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67a67dc163402b91f45d99906b9cd8" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39567 } } }
I20260928 15:57:30.673086 28484 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:30.673084 28485 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:30.674232 28492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:30.678584 28492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:30.687119 28492 catalog_manager.cc:1347] Generated new cluster ID: ab29d41d17b74461bf29d4cdc6827f50
I20260928 15:57:30.687222 28492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:30.704695 28492 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:30.706382 28492 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:30.712942 28492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca67a67dc163402b91f45d99906b9cd8: Generated new TSK 0
I20260928 15:57:30.713708 28492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 15:57:30.735764 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:30.728595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40324 (local address 127.22.125.254:39567)
0928 15:57:30.728743 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:30.728748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:30.729155 (+ 407us) server_negotiation.cc:407] Connection header received
0928 15:57:30.729399 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:30.729403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:30.729465 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:30.729608 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0928 15:57:30.730133 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.730860 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.732259 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.732721 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.732870 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:30.733360 (+ 490us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 15:57:30.735146 (+ 1786us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40727/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 15:57:30.735468 (+ 322us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40324: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40727/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260928 15:57:30.735970 28508 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:30.728434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39567 (local address 127.0.0.1:40324)
0928 15:57:30.729056 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:30.729075 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:30.729221 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:30.729636 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:30.729642 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:30.729656 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0928 15:57:30.729986 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:30.729996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.731005 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.731011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:30.732122 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:30.732133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:30.732913 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:30.732921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:30.733066 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:30.733099 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 15:57:30.735328 (+ 2229us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40727/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 15:57:30.735576 (+ 248us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.125.254:39567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40727/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":342,"threads_started":1}
W20260928 15:57:30.736533 28481 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40324: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40727/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260928 15:57:30.736949 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28411
I20260928 15:57:30.738178 28473 generic_service.cc:196] Checking for leaks (request via RPC)
E20260928 15:57:30.747882 28509 webserver.cc:533] Webserver: sslize failed:
E20260928 15:57:30.747999 28509 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 15:57:30.828379 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28411
2026-09-28T15:57:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (685 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T15:57:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:30Z Disabled control of system clock
I20260928 15:57:30.906520 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:43879
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:43879
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/testchainca.cert with env {}
W20260928 15:57:31.220693 28514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:31.221007 28514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:31.221149 28514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:31.228284 28514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:31.228390 28514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:31.228475 28514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:31.228574 28514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:31.228652 28514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 15:57:31.228741 28514 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 15:57:31.228809 28514 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 15:57:31.238677 28514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:43879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:43879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:31.239948 28514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:31.241230 28514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:31.252871 28519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:31.252931 28520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:31.252954 28522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:31.253876 28514 server_base.cc:1034] running on GCE node
I20260928 15:57:31.254586 28514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:31.255937 28514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:31.257225 28514 hybrid_clock.cc:648] HybridClock initialized: now 1790611051257161 us; error 52 us; skew 500 ppm
I20260928 15:57:31.257579 28514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:57:31.259836 28514 webserver.cc:460] Webserver started at http://127.22.125.254:44901/ using document root <none> and password file <none>
I20260928 15:57:31.260571 28514 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:31.260705 28514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:31.261020 28514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:31.263816 28514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "2698fc29f5ed4ff3beba4f958d70b9fe"
format_stamp: "Formatted at 2026-09-28 15:57:31 on dist-test-slave-9cqd"
server_key: "2ce386c17e965119be6cf48779e8f778"
server_key_iv: "e8490b6fa4eec559db908a1ef9513534"
server_key_version: "encryptionkey@0"
I20260928 15:57:31.264655 28514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "2698fc29f5ed4ff3beba4f958d70b9fe"
format_stamp: "Formatted at 2026-09-28 15:57:31 on dist-test-slave-9cqd"
server_key: "2ce386c17e965119be6cf48779e8f778"
server_key_iv: "e8490b6fa4eec559db908a1ef9513534"
server_key_version: "encryptionkey@0"
I20260928 15:57:31.269482 28514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 15:57:31.272778 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:31.274013 28514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:31.274192 28514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "2698fc29f5ed4ff3beba4f958d70b9fe"
format_stamp: "Formatted at 2026-09-28 15:57:31 on dist-test-slave-9cqd"
server_key: "2ce386c17e965119be6cf48779e8f778"
server_key_iv: "e8490b6fa4eec559db908a1ef9513534"
server_key_version: "encryptionkey@0"
I20260928 15:57:31.274400 28514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:31.297997 28514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:31.298862 28514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:31.299145 28514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:31.316807 28514 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:43879
I20260928 15:57:31.316830 28579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:43879 every 8 connection(s)
I20260928 15:57:31.318171 28514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:31.321818 28580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:31.323355 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28514
I20260928 15:57:31.323575 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:31.324462 23031 external_mini_cluster.cc:1428] Setting key 06c9aceb54bc7b339446dead53c2dd52
I20260928 15:57:31.333895 28580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe: Bootstrap starting.
I20260928 15:57:31.337415 28580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:31.337869 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:31.326364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.22.125.254:43879)
0928 15:57:31.327399 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:31.327409 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:31.327449 (+ 40us) server_negotiation.cc:407] Connection header received
0928 15:57:31.329123 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:31.329145 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:31.329582 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:31.329890 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:31.330477 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:31.331776 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:31.333500 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:31.334230 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:31.334368 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:31.335587 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:31.335594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:31.335606 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:57:31.336204 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:31.336300 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:57:31.336320 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:31.336404 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:31.336650 (+ 246us) server_negotiation.cc:299] Negotiation successful
0928 15:57:31.336954 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":409,"threads_started":1}
I20260928 15:57:31.338480 28580 log.cc:824] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:31.341470 28580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe: No bootstrap required, opened a new log
I20260928 15:57:31.341655 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 15:57:31.348598 28580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43879 } }
I20260928 15:57:31.348984 28580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:31.349085 28580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2698fc29f5ed4ff3beba4f958d70b9fe, State: Initialized, Role: FOLLOWER
I20260928 15:57:31.349587 28580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43879 } }
I20260928 15:57:31.349809 28580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:31.349943 28580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:31.350106 28580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:31.352205 28580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43879 } }
I20260928 15:57:31.352680 28580 leader_election.cc:302] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2698fc29f5ed4ff3beba4f958d70b9fe; no voters:
I20260928 15:57:31.353251 28580 leader_election.cc:288] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:31.353451 28586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:31.354367 28586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [term 1 LEADER]: Becoming Leader. State: Replica: 2698fc29f5ed4ff3beba4f958d70b9fe, State: Running, Role: LEADER
I20260928 15:57:31.354930 28586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43879 } }
I20260928 15:57:31.355300 28580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:31.357951 28587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43879 } } }
I20260928 15:57:31.357981 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2698fc29f5ed4ff3beba4f958d70b9fe. Latest consensus state: current_term: 1 leader_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2698fc29f5ed4ff3beba4f958d70b9fe" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 43879 } } }
I20260928 15:57:31.358439 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:31.358444 28587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:31.360435 28595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:31.365140 28595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:31.373188 28595 catalog_manager.cc:1347] Generated new cluster ID: d1dafc73b1ec4448a92e84d8f771dfc7
I20260928 15:57:31.373299 28595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:31.385129 28595 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:31.386317 28595 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:31.392877 28595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2698fc29f5ed4ff3beba4f958d70b9fe: Generated new TSK 0
I20260928 15:57:31.393620 28595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 15:57:31.439698 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:31.413075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52538 (local address 127.22.125.254:43879)
0928 15:57:31.413268 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:31.413275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:31.413672 (+ 397us) server_negotiation.cc:407] Connection header received
0928 15:57:31.413891 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:31.413902 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:31.413998 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:31.414166 (+ 168us) server_negotiation.cc:226] Negotiated authn=JWT
0928 15:57:31.414557 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:31.415668 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:31.417062 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:31.417760 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:31.417898 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:31.418314 (+ 416us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 15:57:31.439010 (+ 20696us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 15:57:31.439379 (+ 369us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52538: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":62}
W20260928 15:57:31.439764 28610 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:31.412939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:43879 (local address 127.0.0.1:52538)
0928 15:57:31.413579 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:31.413594 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:31.413705 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:31.414167 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:31.414170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:31.414180 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0928 15:57:31.414411 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:31.414418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:31.415804 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:31.415807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:31.416928 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:31.416940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:31.417922 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:31.417926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:31.418047 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:31.418076 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 15:57:31.439255 (+ 21179us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 15:57:31.439474 (+ 219us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.125.254:43879: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":348,"threads_started":1}
W20260928 15:57:31.439867 28584 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52538: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260928 15:57:31.441325 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28514
I20260928 15:57:31.442514 28575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:31.534430 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28514
2026-09-28T15:57:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (687 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (403 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (415 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:32 dist-test-slave-9cqd krb5kdc[28625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:32 dist-test-slave-9cqd krb5kdc[28625](info): set up 2 sockets
Sep 28 15:57:32 dist-test-slave-9cqd krb5kdc[28625](info): commencing operation
krb5kdc: starting...
W20260928 15:57:33.430658 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:33 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:33.637287 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35011
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:35011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:33.965672 28638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:33.966005 28638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:33.966190 28638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:33.973402 28638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:33.973543 28638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:33.973656 28638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:33.973752 28638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:33.973840 28638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:33.983564 28638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:35011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:33.984908 28638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:33.986281 28638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:33.997893 28643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:33.997924 28646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:33.997987 28644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:33.998785 28638 server_base.cc:1034] running on GCE node
I20260928 15:57:33.999570 28638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:34.000979 28638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:34.002244 28638 hybrid_clock.cc:648] HybridClock initialized: now 1790611054002204 us; error 49 us; skew 500 ppm
I20260928 15:57:34.002590 28638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:34 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611054, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:34.005815 28638 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:34.007308 28638 webserver.cc:460] Webserver started at http://127.22.125.254:37537/ using document root <none> and password file <none>
I20260928 15:57:34.007921 28638 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:34.008019 28638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:34.008366 28638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:34.011207 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "cbf5ea688e694d15818e641c9c1db1e7"
format_stamp: "Formatted at 2026-09-28 15:57:34 on dist-test-slave-9cqd"
server_key: "08e8f0ed95b40d239eab2ca045b80cbd"
server_key_iv: "4a1845a5f1c0cabfd50d6fa583f6c0dc"
server_key_version: "encryptionkey@0"
I20260928 15:57:34.011917 28638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "cbf5ea688e694d15818e641c9c1db1e7"
format_stamp: "Formatted at 2026-09-28 15:57:34 on dist-test-slave-9cqd"
server_key: "08e8f0ed95b40d239eab2ca045b80cbd"
server_key_iv: "4a1845a5f1c0cabfd50d6fa583f6c0dc"
server_key_version: "encryptionkey@0"
I20260928 15:57:34.016873 28638 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.002s
I20260928 15:57:34.020167 28653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:34.021483 28638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 15:57:34.021672 28638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "cbf5ea688e694d15818e641c9c1db1e7"
format_stamp: "Formatted at 2026-09-28 15:57:34 on dist-test-slave-9cqd"
server_key: "08e8f0ed95b40d239eab2ca045b80cbd"
server_key_iv: "4a1845a5f1c0cabfd50d6fa583f6c0dc"
server_key_version: "encryptionkey@0"
I20260928 15:57:34.021862 28638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:34.039431 28638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:34.043810 28638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:34.044162 28638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:34.062317 28638 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:35011
I20260928 15:57:34.062353 28704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:35011 every 8 connection(s)
I20260928 15:57:34.063663 28638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:34.066340 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28638
I20260928 15:57:34.066572 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:34.067247 23031 external_mini_cluster.cc:1428] Setting key 22c2dac7bf9e2709b481068a6f922697
I20260928 15:57:34.067462 28705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:34.081525 28705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7: Bootstrap starting.
Sep 28 15:57:34 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:34.085363 28705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:34.086767 28705 log.cc:824] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:34.090440 28705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7: No bootstrap required, opened a new log
I20260928 15:57:34.091807 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:34.069199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.22.125.254:35011)
0928 15:57:34.070015 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:34.070029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:34.070074 (+ 45us) server_negotiation.cc:407] Connection header received
0928 15:57:34.072373 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:34.072401 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:34.072909 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:34.073341 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:34.074633 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:34.076630 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:34.077835 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:34.078935 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:34.079116 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:34.083102 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:34.083113 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:34.083132 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:34.083179 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:34.087322 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:34.088224 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:34.088231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:34.088237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:34.088436 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:34.088797 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:34.088802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:34.088804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:34.089476 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:34.089784 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:34.090073 (+ 289us) server_negotiation.cc:299] Negotiation successful
0928 15:57:34.090494 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":311,"threads_started":1}
I20260928 15:57:34.098994 28705 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbf5ea688e694d15818e641c9c1db1e7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35011 } }
I20260928 15:57:34.099360 28705 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:34.099435 28705 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf5ea688e694d15818e641c9c1db1e7, State: Initialized, Role: FOLLOWER
I20260928 15:57:34.099895 28705 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbf5ea688e694d15818e641c9c1db1e7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35011 } }
I20260928 15:57:34.100168 28705 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:34.100296 28705 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:34.100466 28705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:34.102664 28705 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbf5ea688e694d15818e641c9c1db1e7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35011 } }
I20260928 15:57:34.103145 28705 leader_election.cc:302] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbf5ea688e694d15818e641c9c1db1e7; no voters:
I20260928 15:57:34.103711 28705 leader_election.cc:288] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:34.103832 28710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:34.104570 28710 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [term 1 LEADER]: Becoming Leader. State: Replica: cbf5ea688e694d15818e641c9c1db1e7, State: Running, Role: LEADER
I20260928 15:57:34.105046 28710 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbf5ea688e694d15818e641c9c1db1e7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35011 } }
I20260928 15:57:34.105713 28705 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:34.107790 28711 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbf5ea688e694d15818e641c9c1db1e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbf5ea688e694d15818e641c9c1db1e7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35011 } } }
I20260928 15:57:34.108228 28711 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:34.107779 28712 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbf5ea688e694d15818e641c9c1db1e7. Latest consensus state: current_term: 1 leader_uuid: "cbf5ea688e694d15818e641c9c1db1e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbf5ea688e694d15818e641c9c1db1e7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35011 } } }
I20260928 15:57:34.109470 28712 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:34.110612 28717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:34.115312 28717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:34.124356 28717 catalog_manager.cc:1347] Generated new cluster ID: 08acb2567f6546ad9bd18e196b68158f
I20260928 15:57:34.124471 28717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:34.150230 28717 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:34.151608 28717 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:34.167634 28717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbf5ea688e694d15818e641c9c1db1e7: Generated new TSK 0
I20260928 15:57:34.168543 28717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:57:34.215597 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35011
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:34.540830 28731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:34.541095 28731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:34.541277 28731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:34.548458 28731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:34.548581 28731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:34.548765 28731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:34.558218 28731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:34.559500 28731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:34.560813 28731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:34.573546 28737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:34.573585 28736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:34.573499 28739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:34.574231 28731 server_base.cc:1034] running on GCE node
I20260928 15:57:34.574834 28731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:34.575688 28731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:34.576879 28731 hybrid_clock.cc:648] HybridClock initialized: now 1790611054576818 us; error 54 us; skew 500 ppm
I20260928 15:57:34.577198 28731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:34 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611054, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:34.580241 28731 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:57:34.581557 28731 webserver.cc:460] Webserver started at http://127.22.125.193:33151/ using document root <none> and password file <none>
I20260928 15:57:34.582214 28731 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:34.582360 28731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:34.582690 28731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:34.585558 28731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "b1da7649034142059f7e43dcea9e815c"
format_stamp: "Formatted at 2026-09-28 15:57:34 on dist-test-slave-9cqd"
server_key: "15963d9b85629971bdda456e1e70dd63"
server_key_iv: "d3ccd534314c0e7bbc2c78a113b55856"
server_key_version: "encryptionkey@0"
I20260928 15:57:34.586336 28731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "b1da7649034142059f7e43dcea9e815c"
format_stamp: "Formatted at 2026-09-28 15:57:34 on dist-test-slave-9cqd"
server_key: "15963d9b85629971bdda456e1e70dd63"
server_key_iv: "d3ccd534314c0e7bbc2c78a113b55856"
server_key_version: "encryptionkey@0"
I20260928 15:57:34.591579 28731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 15:57:34.594987 28746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:34.596246 28731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:34.596405 28731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "b1da7649034142059f7e43dcea9e815c"
format_stamp: "Formatted at 2026-09-28 15:57:34 on dist-test-slave-9cqd"
server_key: "15963d9b85629971bdda456e1e70dd63"
server_key_iv: "d3ccd534314c0e7bbc2c78a113b55856"
server_key_version: "encryptionkey@0"
I20260928 15:57:34.596598 28731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:34.610018 28731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:34.614154 28731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:34.614498 28731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:34.615401 28731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:34.616918 28731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:34.617014 28731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:34.617139 28731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:34.617225 28731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260928 15:57:34.651077 28731 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:41521
I20260928 15:57:34.651122 28861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:41521 every 8 connection(s)
I20260928 15:57:34.652391 28731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:34.655452 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28731
I20260928 15:57:34.655697 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:34.656404 23031 external_mini_cluster.cc:1428] Setting key 3fbc17b1af48b35b97f06f44345af749
Sep 28 15:57:34 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611054, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:34.673869 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:34.654892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:33207 (local address 127.22.125.254:35011)
0928 15:57:34.655442 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:34.655448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:34.656575 (+ 1127us) server_negotiation.cc:407] Connection header received
0928 15:57:34.658622 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:34.658628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:34.658705 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:34.658824 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:34.660314 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:34.661454 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:34.662929 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:34.663652 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:34.663792 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:34.668115 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:34.668127 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:34.668131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:34.668164 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:34.670714 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:34.671448 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:34.671452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:34.671455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:34.671571 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:34.672020 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:34.672025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:34.672030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:34.672433 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:34.672587 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:34.673248 (+ 661us) server_negotiation.cc:299] Negotiation successful
0928 15:57:34.673481 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":248,"threads_started":1}
I20260928 15:57:34.674809 28865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:34.655496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35011 (local address 127.22.125.193:33207)
0928 15:57:34.656421 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:34.656468 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:34.658372 (+ 1904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:34.659018 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:34.659027 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:34.659541 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:34.660153 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:34.660172 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:34.661583 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:34.661587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:34.662784 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:34.662795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:34.663843 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:34.663847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:34.664126 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:34.664898 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:34.664933 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:34.667835 (+ 2902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:34.670864 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:34.670872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:34.670888 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:34.671310 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:34.671683 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:34.671689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:34.671692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:34.671903 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:34.672562 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:34.672570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:34.672906 (+ 336us) client_negotiation.cc:769] Sending connection context
0928 15:57:34.673155 (+ 249us) client_negotiation.cc:240] Negotiation successful
0928 15:57:34.673695 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":407,"threads_started":1}
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:57:34.676435 28862 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35011
I20260928 15:57:34.676738 28862 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:34.677512 28862 heartbeater.cc:511] Master 127.22.125.254:35011 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:57:34.679709 28670 ts_manager.cc:194] Registered new tserver with Master: b1da7649034142059f7e43dcea9e815c (127.22.125.193:41521)
I20260928 15:57:34.681222 28670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:33207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:57:34.702593 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35011
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:35.025658 28868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:35.025929 28868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:35.026074 28868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:35.033360 28868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:35.033463 28868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:35.033571 28868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:35.043035 28868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:35.044277 28868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:35.045512 28868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:35.058455 28875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:35.058530 28874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:35.058493 28877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:35.059259 28868 server_base.cc:1034] running on GCE node
I20260928 15:57:35.059815 28868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:35.060842 28868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:35.062059 28868 hybrid_clock.cc:648] HybridClock initialized: now 1790611055061991 us; error 48 us; skew 500 ppm
I20260928 15:57:35.062392 28868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.065659 28868 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:57:35.067175 28868 webserver.cc:460] Webserver started at http://127.22.125.194:39287/ using document root <none> and password file <none>
I20260928 15:57:35.067831 28868 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:35.067987 28868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:35.068305 28868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:35.071064 28868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "5949c6db3f554037a83d77240b4bf9d3"
format_stamp: "Formatted at 2026-09-28 15:57:35 on dist-test-slave-9cqd"
server_key: "17d23225e30420c84b19bbbe64c4ba7c"
server_key_iv: "33a93b7e97bdb7dbe5c3de41a13cff74"
server_key_version: "encryptionkey@0"
I20260928 15:57:35.071818 28868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "5949c6db3f554037a83d77240b4bf9d3"
format_stamp: "Formatted at 2026-09-28 15:57:35 on dist-test-slave-9cqd"
server_key: "17d23225e30420c84b19bbbe64c4ba7c"
server_key_iv: "33a93b7e97bdb7dbe5c3de41a13cff74"
server_key_version: "encryptionkey@0"
I20260928 15:57:35.076910 28868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 15:57:35.080214 28884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:35.081485 28868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 15:57:35.081663 28868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "5949c6db3f554037a83d77240b4bf9d3"
format_stamp: "Formatted at 2026-09-28 15:57:35 on dist-test-slave-9cqd"
server_key: "17d23225e30420c84b19bbbe64c4ba7c"
server_key_iv: "33a93b7e97bdb7dbe5c3de41a13cff74"
server_key_version: "encryptionkey@0"
I20260928 15:57:35.081868 28868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:35.099375 28868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:35.103472 28868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:35.103758 28868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:35.104710 28868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:35.106324 28868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:35.106463 28868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260928 15:57:35.106611 28868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:35.106719 28868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:35.139220 28868 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:41389
I20260928 15:57:35.139266 28997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:41389 every 8 connection(s)
I20260928 15:57:35.140511 28868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:35.141806 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 28868
I20260928 15:57:35.142009 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:35.142449 23031 external_mini_cluster.cc:1428] Setting key 3df8180fc92e0ae2613391944eee9056
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:35.163008 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.143255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:42253 (local address 127.22.125.254:35011)
0928 15:57:35.143455 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.143461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.145055 (+ 1594us) server_negotiation.cc:407] Connection header received
0928 15:57:35.147883 (+ 2828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.147887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.147950 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.148034 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.150121 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.150802 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.152738 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.153216 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.153314 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.157363 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.157368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.157371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.157402 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.160011 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.160772 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.160778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.160781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.160922 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.161348 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.161357 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.161360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.161618 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.161776 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:35.162440 (+ 664us) server_negotiation.cc:299] Negotiation successful
0928 15:57:35.162672 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 15:57:35.163991 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.143866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35011 (local address 127.22.125.194:42253)
0928 15:57:35.144887 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:35.144939 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:35.147606 (+ 2667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:35.148322 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:35.148337 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:35.149138 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:35.149956 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:35.149982 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.150978 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.150986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:35.152596 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:35.152608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.153460 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.153469 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:35.153803 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.154508 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:35.154547 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:35.157147 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:35.160206 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:35.160214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:35.160230 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:35.160643 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:35.161034 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:35.161039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:35.161042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:35.161226 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:35.161756 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:35.161763 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:35.162091 (+ 328us) client_negotiation.cc:769] Sending connection context
0928 15:57:35.162353 (+ 262us) client_negotiation.cc:240] Negotiation successful
0928 15:57:35.162843 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":372,"threads_started":1}
I20260928 15:57:35.165417 28998 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35011
I20260928 15:57:35.165771 28998 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
I20260928 15:57:35.166502 28998 heartbeater.cc:511] Master 127.22.125.254:35011 requested a full tablet report, sending...
I20260928 15:57:35.168075 28670 ts_manager.cc:194] Registered new tserver with Master: 5949c6db3f554037a83d77240b4bf9d3 (127.22.125.194:41389)
I20260928 15:57:35.169282 28670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:42253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:57:35.189002 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35011
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:35.504134 29003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:35.504407 29003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:35.504505 29003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:35.511817 29003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:35.511930 29003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:35.512086 29003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:35.521960 29003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:35.523161 29003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:35.524518 29003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:35.537832 29009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:35.537879 29011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:35.537899 29008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:35.545413 29003 server_base.cc:1034] running on GCE node
I20260928 15:57:35.546089 29003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:35.547050 29003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:35.548285 29003 hybrid_clock.cc:648] HybridClock initialized: now 1790611055548209 us; error 52 us; skew 500 ppm
I20260928 15:57:35.548635 29003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.551599 29003 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:57:35.553062 29003 webserver.cc:460] Webserver started at http://127.22.125.195:43411/ using document root <none> and password file <none>
I20260928 15:57:35.553711 29003 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:35.553810 29003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:35.554069 29003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:35.556895 29003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "d80193c3f88645488bb73c2776916324"
format_stamp: "Formatted at 2026-09-28 15:57:35 on dist-test-slave-9cqd"
server_key: "9a77d1b1fbed6018cf878d08f8982cfb"
server_key_iv: "e17cba8ff557292047011c049806ef19"
server_key_version: "encryptionkey@0"
I20260928 15:57:35.557634 29003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "d80193c3f88645488bb73c2776916324"
format_stamp: "Formatted at 2026-09-28 15:57:35 on dist-test-slave-9cqd"
server_key: "9a77d1b1fbed6018cf878d08f8982cfb"
server_key_iv: "e17cba8ff557292047011c049806ef19"
server_key_version: "encryptionkey@0"
I20260928 15:57:35.562598 29003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:35.565788 29018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:35.567060 29003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:57:35.567219 29003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "d80193c3f88645488bb73c2776916324"
format_stamp: "Formatted at 2026-09-28 15:57:35 on dist-test-slave-9cqd"
server_key: "9a77d1b1fbed6018cf878d08f8982cfb"
server_key_iv: "e17cba8ff557292047011c049806ef19"
server_key_version: "encryptionkey@0"
I20260928 15:57:35.567420 29003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:35.592221 29003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:35.596369 29003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:35.596649 29003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:35.597572 29003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:35.598836 29003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:35.598946 29003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:35.599092 29003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:35.599162 29003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:35.633668 29003 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:45281
I20260928 15:57:35.633718 29131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:45281 every 8 connection(s)
I20260928 15:57:35.634959 29003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:35.637624 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29003
I20260928 15:57:35.637826 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:35.638270 23031 external_mini_cluster.cc:1428] Setting key b05dfb9bd1c74a32e5ada722d2b206d1
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:35.657294 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.637694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:55795 (local address 127.22.125.254:35011)
0928 15:57:35.638662 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.638666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.639269 (+ 603us) server_negotiation.cc:407] Connection header received
0928 15:57:35.641627 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.641631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.641697 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.641778 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.643178 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.643910 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.645561 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.645974 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.646074 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.649476 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.649481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.649484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.649516 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.652492 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.653375 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.653385 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.653388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.653908 (+ 520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.654432 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.654436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.654438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.654715 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.654829 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:35.656738 (+ 1909us) server_negotiation.cc:299] Negotiation successful
0928 15:57:35.656968 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843}
I20260928 15:57:35.657367 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.638216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35011 (local address 127.22.125.195:55795)
0928 15:57:35.639063 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:35.639140 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:35.641307 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:35.641949 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:35.641957 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:35.642479 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:35.643035 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:35.643053 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.644080 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.644086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:35.645330 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:35.645346 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.646170 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.646175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:35.646381 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.647078 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:35.647101 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:35.649289 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:35.652624 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:35.652634 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:35.652649 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:35.653219 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:35.654050 (+ 831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:35.654055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:35.654059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:35.654299 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:35.654832 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:35.654839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:35.655220 (+ 381us) client_negotiation.cc:769] Sending connection context
0928 15:57:35.655507 (+ 287us) client_negotiation.cc:240] Negotiation successful
0928 15:57:35.656014 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":357,"threads_started":1}
I20260928 15:57:35.658653 29132 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35011
I20260928 15:57:35.658942 29132 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:35.659650 29132 heartbeater.cc:511] Master 127.22.125.254:35011 requested a full tablet report, sending...
I20260928 15:57:35.661026 28670 ts_manager.cc:194] Registered new tserver with Master: d80193c3f88645488bb73c2776916324 (127.22.125.195:45281)
I20260928 15:57:35.661799 28670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:55795
I20260928 15:57:35.666023 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:57:35.684206 28862 heartbeater.cc:503] Master 127.22.125.254:35011 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/test-admin.keytab.
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.693202 23031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 15:57:35.698151 29139 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:35.695807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.22.125.193:41521)
0928 15:57:35.696469 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.696476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.696506 (+ 30us) server_negotiation.cc:407] Connection header received
0928 15:57:35.696565 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.696571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.696780 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.696909 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.697777 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58684: BlockingRecv error: recv got EOF from 127.0.0.1:58684 (error 108)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":355,"threads_started":1}
W20260928 15:57:35.698510 23031 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5cc' of type FILE
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.700160 23031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.701607 23031 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:35.717165 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.703177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58692 (local address 127.22.125.193:41521)
0928 15:57:35.703384 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.703391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.703642 (+ 251us) server_negotiation.cc:407] Connection header received
0928 15:57:35.703844 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.703849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.703923 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.704027 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.705462 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.706573 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.707799 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.708428 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.708531 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.711826 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.711837 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.711854 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.711885 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.714709 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.715314 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.715321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.715323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.715440 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.715749 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.715751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.715753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.716153 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.716384 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:35.716633 (+ 249us) server_negotiation.cc:299] Negotiation successful
0928 15:57:35.716878 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260928 15:57:35.722625 29139 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:35.721007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.22.125.193:41521)
0928 15:57:35.721159 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.721162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.721503 (+ 341us) server_negotiation.cc:407] Connection header received
0928 15:57:35.721620 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.721623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.721686 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.721760 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.722355 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58702: BlockingRecv error: recv got EOF from 127.0.0.1:58702 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260928 15:57:35.722796 23031 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5cc' of type FILE
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.724332 23031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.725653 23031 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:35.739051 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.727094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.22.125.193:41521)
0928 15:57:35.727287 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.727295 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.727486 (+ 191us) server_negotiation.cc:407] Connection header received
0928 15:57:35.727671 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.727680 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.727734 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.727830 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.728821 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.729508 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.730686 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.731120 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.731254 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.734716 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.734721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.734724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.734753 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.737098 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.737629 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.737633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.737635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.737743 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.738056 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.738059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.738061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.738291 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.738470 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:35.738684 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 15:57:35.738830 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260928 15:57:35.743263 29139 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:35.741686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.22.125.193:41521)
0928 15:57:35.741881 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.741887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.742059 (+ 172us) server_negotiation.cc:407] Connection header received
0928 15:57:35.742234 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.742239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.742308 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.742410 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.743055 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58726: BlockingRecv error: recv got EOF from 127.0.0.1:58726 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260928 15:57:35.743460 23031 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790611005938867-23031-0/krb5kdc/krb5cc' of type FILE
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.744870 23031 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:35.746138 23031 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:35.759842 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.747240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.22.125.193:41521)
0928 15:57:35.747447 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.747454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.747703 (+ 249us) server_negotiation.cc:407] Connection header received
0928 15:57:35.747872 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.747879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.747942 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.748038 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.748978 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.749676 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.750882 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.751294 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.751436 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.754811 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.754816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.754819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.754848 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.757279 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.757924 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.757931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.757934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.758088 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.758448 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.758454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.758457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.758737 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.759060 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:35.759331 (+ 271us) server_negotiation.cc:299] Negotiation successful
0928 15:57:35.759543 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 15:57:35.761354 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28731
I20260928 15:57:35.772440 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.761992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.22.125.193:41521)
0928 15:57:35.762129 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.762133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.762635 (+ 502us) server_negotiation.cc:407] Connection header received
0928 15:57:35.762759 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.762762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.762811 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.762885 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.763674 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.764387 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.765529 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.765920 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.766012 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.767831 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.767836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.767839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.767865 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.770156 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.770792 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.770796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.770798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.770924 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.771217 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.771220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.771222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.771445 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.771628 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:35.771935 (+ 307us) server_negotiation.cc:299] Negotiation successful
0928 15:57:35.772176 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 15:57:35.773001 28857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:35.856630 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28731
I20260928 15:57:35.872785 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28868
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:57:35.887835 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.873651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.22.125.194:41389)
0928 15:57:35.874229 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.874236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.874262 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:57:35.874332 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.874345 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.874566 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.874698 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.875635 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.876742 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.877907 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.878443 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.878537 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.882440 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.882451 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.882465 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.882495 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.885196 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.885915 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.885925 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.885928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.886067 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.886407 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.886411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.886414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.886763 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.886990 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:35.887269 (+ 279us) server_negotiation.cc:299] Negotiation successful
0928 15:57:35.887524 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":312,"threads_started":1}
I20260928 15:57:35.889104 28993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:35.970094 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28868
I20260928 15:57:35.985605 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29003
Sep 28 15:57:35 dist-test-slave-9cqd krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:57:36.000715 29155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:35.986502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.22.125.195:45281)
0928 15:57:35.987149 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:35.987155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:35.987184 (+ 29us) server_negotiation.cc:407] Connection header received
0928 15:57:35.987246 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:35.987251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:35.987481 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:35.987616 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:35.988689 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.989709 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.990856 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:35.991404 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:35.991496 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:35.995472 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:35.995483 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:35.995497 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:35.995535 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:35.998403 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.998981 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.998988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.998994 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.999112 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:35.999430 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:35.999433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:35.999435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:35.999761 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:35.999981 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:36.000233 (+ 252us) server_negotiation.cc:299] Negotiation successful
0928 15:57:36.000440 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":341,"threads_started":1}
I20260928 15:57:36.001969 29127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:36.085435 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29003
I20260928 15:57:36.100736 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 28638
I20260928 15:57:36.101914 28700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:36.205214 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 28638
2026-09-28T15:57:36Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3858 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:36 dist-test-slave-9cqd krb5kdc[29160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:36 dist-test-slave-9cqd krb5kdc[29160](info): set up 2 sockets
Sep 28 15:57:36 dist-test-slave-9cqd krb5kdc[29160](info): commencing operation
krb5kdc: starting...
W20260928 15:57:37.286476 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.002s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:37 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab.
Entry for principal oryx/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab.
I20260928 15:57:37.484822 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:37163
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab
--principal=oryx/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:37163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:37.797995 29173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:37.798321 29173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:37.798478 29173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:37.805536 29173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:37.805675 29173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:37.805773 29173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:37.805863 29173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:37.805940 29173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:37.815965 29173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:37163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab
--principal=oryx/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:37163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:37.817303 29173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:37.818608 29173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:37.830005 29179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:37.830099 29178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:37.830050 29181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:37.831120 29173 server_base.cc:1034] running on GCE node
I20260928 15:57:37.831827 29173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:37.833212 29173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:37.834425 29173 hybrid_clock.cc:648] HybridClock initialized: now 1790611057834380 us; error 41 us; skew 500 ppm
I20260928 15:57:37.834712 29173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:37 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:37.841615 29173 init.cc:377] Logged in from keytab as oryx/127.22.125.254@KRBTEST.COM (short username oryx)
I20260928 15:57:37.843080 29173 webserver.cc:460] Webserver started at http://127.22.125.254:34867/ using document root <none> and password file <none>
I20260928 15:57:37.843775 29173 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:37.843922 29173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:37.844282 29173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:37.847262 29173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "6991a5163c3e404fb77e798a3f36fee3"
format_stamp: "Formatted at 2026-09-28 15:57:37 on dist-test-slave-9cqd"
server_key: "ab4306cd2e139a351377924e99ca809b"
server_key_iv: "ce21681cad3b01f02bf8a06b68dc9042"
server_key_version: "encryptionkey@0"
I20260928 15:57:37.848003 29173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "6991a5163c3e404fb77e798a3f36fee3"
format_stamp: "Formatted at 2026-09-28 15:57:37 on dist-test-slave-9cqd"
server_key: "ab4306cd2e139a351377924e99ca809b"
server_key_iv: "ce21681cad3b01f02bf8a06b68dc9042"
server_key_version: "encryptionkey@0"
I20260928 15:57:37.853030 29173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 15:57:37.856623 29188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:37.857885 29173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:37.858076 29173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "6991a5163c3e404fb77e798a3f36fee3"
format_stamp: "Formatted at 2026-09-28 15:57:37 on dist-test-slave-9cqd"
server_key: "ab4306cd2e139a351377924e99ca809b"
server_key_iv: "ce21681cad3b01f02bf8a06b68dc9042"
server_key_version: "encryptionkey@0"
I20260928 15:57:37.858275 29173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:37.872426 29173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:37.875998 29173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:37.876304 29173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:37.893666 29173 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:37163
I20260928 15:57:37.893703 29240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:37163 every 8 connection(s)
I20260928 15:57:37.894923 29173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:37.898435 29241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:37.903581 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29173
I20260928 15:57:37.903825 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:37.904469 23031 external_mini_cluster.cc:1428] Setting key 81692ce70439b01f395db864b3e0aab1
I20260928 15:57:37.909142 29241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: Bootstrap starting.
I20260928 15:57:37.912789 29241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:37.914027 29241 log.cc:824] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:37.916706 29241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: No bootstrap required, opened a new log
Sep 28 15:57:37 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:37.924363 29241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } }
I20260928 15:57:37.924790 29241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:37.924908 29241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6991a5163c3e404fb77e798a3f36fee3, State: Initialized, Role: FOLLOWER
I20260928 15:57:37.925580 29241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } }
I20260928 15:57:37.925863 29241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:37.926000 29241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:37.926192 29241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:37.927461 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:37.906355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.22.125.254:37163)
0928 15:57:37.907229 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:37.907246 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:37.907296 (+ 50us) server_negotiation.cc:407] Connection header received
0928 15:57:37.908955 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:37.908976 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:37.909379 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:37.909686 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:37.911555 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:37.912910 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:37.914762 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:37.915454 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:37.915648 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:37.919626 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:37.919633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:37.919646 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:37.919681 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:37.922839 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:37.923559 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:37.923573 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:37.923579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:37.923738 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:37.924189 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:37.924194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:37.924197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:37.924907 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:37.925196 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:37.925667 (+ 471us) server_negotiation.cc:299] Negotiation successful
0928 15:57:37.926097 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":404,"threads_started":1}
I20260928 15:57:37.929328 29241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } }
I20260928 15:57:37.929800 29241 leader_election.cc:302] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6991a5163c3e404fb77e798a3f36fee3; no voters:
I20260928 15:57:37.930469 29241 leader_election.cc:288] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:37.930526 29246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:37.931257 29246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 1 LEADER]: Becoming Leader. State: Replica: 6991a5163c3e404fb77e798a3f36fee3, State: Running, Role: LEADER
I20260928 15:57:37.931803 29246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } }
I20260928 15:57:37.932511 29241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:37.934748 29248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6991a5163c3e404fb77e798a3f36fee3. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } }
I20260928 15:57:37.934722 29247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } }
I20260928 15:57:37.935164 29248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:37.935236 29247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:37.936192 29254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:37.943384 29254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:37.956597 29254 catalog_manager.cc:1347] Generated new cluster ID: 6aa39b3fca66401e9cb6881c23cd4302
I20260928 15:57:37.956705 29254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:37.964157 29254 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:37.965322 29254 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:37.976939 29254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: Generated new TSK 0
I20260928 15:57:37.977669 29254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab.
Entry for principal oryx/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab.
I20260928 15:57:38.030064 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37163
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab
--principal=oryx/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:38.350695 29267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:38.350967 29267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:38.351059 29267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:38.358367 29267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:38.358469 29267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:38.358665 29267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:38.368671 29267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab
--principal=oryx/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:38.369935 29267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:38.371228 29267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:38.384300 29272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:38.384352 29275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:38.384404 29273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:38.385160 29267 server_base.cc:1034] running on GCE node
I20260928 15:57:38.385679 29267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:38.386571 29267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:38.387794 29267 hybrid_clock.cc:648] HybridClock initialized: now 1790611058387740 us; error 54 us; skew 500 ppm
I20260928 15:57:38.388171 29267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:38 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:38.391419 29267 init.cc:377] Logged in from keytab as oryx/127.22.125.193@KRBTEST.COM (short username oryx)
I20260928 15:57:38.392932 29267 webserver.cc:460] Webserver started at http://127.22.125.193:45313/ using document root <none> and password file <none>
I20260928 15:57:38.393611 29267 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:38.393747 29267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:38.394061 29267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:38.397037 29267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "f4c37d8527da493287e4d0702b9600fc"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "347a564bfe00f50e6cb47eabfdf4cdb5"
server_key_iv: "5cb6be2544ccf1e7501526418b458cf3"
server_key_version: "encryptionkey@0"
I20260928 15:57:38.397854 29267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "f4c37d8527da493287e4d0702b9600fc"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "347a564bfe00f50e6cb47eabfdf4cdb5"
server_key_iv: "5cb6be2544ccf1e7501526418b458cf3"
server_key_version: "encryptionkey@0"
I20260928 15:57:38.403003 29267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 15:57:38.406347 29282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:38.407625 29267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 15:57:38.407788 29267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "f4c37d8527da493287e4d0702b9600fc"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "347a564bfe00f50e6cb47eabfdf4cdb5"
server_key_iv: "5cb6be2544ccf1e7501526418b458cf3"
server_key_version: "encryptionkey@0"
I20260928 15:57:38.408102 29267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:38.419512 29267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:38.423382 29267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:38.423671 29267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:38.424650 29267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:38.426177 29267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:38.426282 29267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:38.426431 29267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:38.426504 29267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:38.457693 29267 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:42437
I20260928 15:57:38.457729 29395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:42437 every 8 connection(s)
I20260928 15:57:38.458969 29267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:38.460381 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29267
I20260928 15:57:38.460613 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:38.461233 23031 external_mini_cluster.cc:1428] Setting key 1e507c61d42adf24469e5481d7dee79f
Sep 28 15:57:38 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:38.483145 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:38.461688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:37995 (local address 127.22.125.254:37163)
0928 15:57:38.462134 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:38.462140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:38.463473 (+ 1333us) server_negotiation.cc:407] Connection header received
0928 15:57:38.466351 (+ 2878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:38.466355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:38.466418 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:38.466512 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:38.468612 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.469367 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.471357 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.471752 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.471912 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:38.476020 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:38.476032 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:38.476063 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:38.476119 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:38.479913 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:38.480718 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:38.480722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:38.480724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:38.480839 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:38.481350 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:38.481353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:38.481355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:38.481626 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:38.481737 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:38.482720 (+ 983us) server_negotiation.cc:299] Negotiation successful
0928 15:57:38.482895 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":213,"threads_started":1}
I20260928 15:57:38.484779 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:38.462406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:37995)
0928 15:57:38.463290 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:38.463350 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:38.466042 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:38.466775 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:38.466790 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:38.467557 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:38.468422 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:38.468451 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.469522 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.469531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:38.471194 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:38.471211 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.472003 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.472010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:38.472371 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:38.473098 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:38.473132 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:38.475768 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:38.480104 (+ 4336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:38.480112 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:38.480125 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:38.480553 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:38.480956 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:38.480964 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:38.480970 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:38.481234 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:38.481766 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:38.481776 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:38.482275 (+ 499us) client_negotiation.cc:769] Sending connection context
0928 15:57:38.482649 (+ 374us) client_negotiation.cc:240] Negotiation successful
0928 15:57:38.483236 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":346,"threads_started":1}
WARNING: no policy specified for oryx/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:57:38.486397 29396 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37163
I20260928 15:57:38.486722 29396 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:38.487504 29396 heartbeater.cc:511] Master 127.22.125.254:37163 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.194@KRBTEST.COM" created.
I20260928 15:57:38.490046 29205 ts_manager.cc:194] Registered new tserver with Master: f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437)
I20260928 15:57:38.492219 29205 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:37995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab.
Entry for principal oryx/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab.
I20260928 15:57:38.512753 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37163
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab
--principal=oryx/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:38.832401 29402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:38.832727 29402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:38.832902 29402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:38.840835 29402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:38.840991 29402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:38.841120 29402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:38.851140 29402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab
--principal=oryx/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:38.852423 29402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:38.853741 29402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:38.867022 29407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:38.866981 29410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:38.867081 29408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:38.868124 29402 server_base.cc:1034] running on GCE node
I20260928 15:57:38.868736 29402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:38.869653 29402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:38.870894 29402 hybrid_clock.cc:648] HybridClock initialized: now 1790611058870854 us; error 51 us; skew 500 ppm
I20260928 15:57:38.871215 29402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:38 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:38.874425 29402 init.cc:377] Logged in from keytab as oryx/127.22.125.194@KRBTEST.COM (short username oryx)
I20260928 15:57:38.875916 29402 webserver.cc:460] Webserver started at http://127.22.125.194:33727/ using document root <none> and password file <none>
I20260928 15:57:38.876631 29402 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:38.876777 29402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:38.877067 29402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:38.879968 29402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "718611922ca04e478eef8495c05fb4f7"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "fc3e089c72443ba437893cac9d7d3999"
server_key_iv: "0e4e6be711dc15071ef6003717fd38df"
server_key_version: "encryptionkey@0"
I20260928 15:57:38.880818 29402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "718611922ca04e478eef8495c05fb4f7"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "fc3e089c72443ba437893cac9d7d3999"
server_key_iv: "0e4e6be711dc15071ef6003717fd38df"
server_key_version: "encryptionkey@0"
I20260928 15:57:38.886004 29402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:38.889495 29417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:38.890801 29402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:57:38.890982 29402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "718611922ca04e478eef8495c05fb4f7"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "fc3e089c72443ba437893cac9d7d3999"
server_key_iv: "0e4e6be711dc15071ef6003717fd38df"
server_key_version: "encryptionkey@0"
I20260928 15:57:38.891192 29402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:38.903326 29402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:38.907378 29402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:38.907662 29402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:38.908886 29402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:38.910132 29402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:38.910238 29402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:38.910383 29402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:38.910454 29402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:38.943413 29402 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:34925
I20260928 15:57:38.943454 29533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:34925 every 8 connection(s)
I20260928 15:57:38.944752 29402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:38.953094 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29402
I20260928 15:57:38.953307 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:38.953863 23031 external_mini_cluster.cc:1428] Setting key d61422b6586e118e1da31686b75713b3
Sep 28 15:57:38 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611058, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:38.966373 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:38.947352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:48623 (local address 127.22.125.254:37163)
0928 15:57:38.947544 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:38.947549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:38.948742 (+ 1193us) server_negotiation.cc:407] Connection header received
0928 15:57:38.950445 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:38.950450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:38.950517 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:38.950608 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:38.952015 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.952815 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.955039 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.955606 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.955760 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:38.959676 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:38.959685 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:38.959692 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:38.959739 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:38.963230 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:38.964174 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:38.964178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:38.964180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:38.964303 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:38.964774 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:38.964777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:38.964779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:38.965035 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:38.965157 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:38.965971 (+ 814us) server_negotiation.cc:299] Negotiation successful
0928 15:57:38.966149 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 15:57:38.967880 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:38.947835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:48623)
0928 15:57:38.948576 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:38.948612 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:38.950191 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:38.950813 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:38.950820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:38.951306 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:38.951860 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:38.951878 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.952967 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.952973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:38.954880 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:38.954902 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:38.955833 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:38.955847 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:38.956113 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:38.956802 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:38.956835 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:38.959460 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:38.963374 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:38.963384 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:38.963402 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:38.964064 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:38.964414 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:38.964421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:38.964428 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:38.964666 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:38.965156 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:38.965167 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:38.965590 (+ 423us) client_negotiation.cc:769] Sending connection context
0928 15:57:38.965884 (+ 294us) client_negotiation.cc:240] Negotiation successful
0928 15:57:38.966454 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":321,"threads_started":1}
I20260928 15:57:38.969347 29534 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37163
I20260928 15:57:38.969650 29534 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:38.970340 29534 heartbeater.cc:511] Master 127.22.125.254:37163 requested a full tablet report, sending...
I20260928 15:57:38.971725 29205 ts_manager.cc:194] Registered new tserver with Master: 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925)
I20260928 15:57:38.972582 29205 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:48623
WARNING: no policy specified for oryx/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab.
Entry for principal oryx/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab.
I20260928 15:57:39.002539 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37163
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab
--principal=oryx/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:39.317615 29539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:39.317869 29539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:39.317958 29539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:39.324838 29539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:39.324936 29539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:39.325042 29539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:39.334365 29539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab
--principal=oryx/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:37163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:39.335527 29539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:39.336789 29539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:39.350014 29544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:39.349942 29547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:39.349985 29545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:39.350641 29539 server_base.cc:1034] running on GCE node
I20260928 15:57:39.351239 29539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:39.352227 29539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:39.353454 29539 hybrid_clock.cc:648] HybridClock initialized: now 1790611059353410 us; error 54 us; skew 500 ppm
I20260928 15:57:39.353760 29539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:39 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:39.356840 29539 init.cc:377] Logged in from keytab as oryx/127.22.125.195@KRBTEST.COM (short username oryx)
I20260928 15:57:39.358210 29539 webserver.cc:460] Webserver started at http://127.22.125.195:40457/ using document root <none> and password file <none>
I20260928 15:57:39.358893 29539 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:39.359027 29539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:39.359335 29539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:39.362432 29539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "d3db5ab25c94440a80c3ab45878cd12a"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "4b69345ff1176f017e3d2223bf5abdc6"
server_key_iv: "aca90b6c46218393742c79d037d7ef71"
server_key_version: "encryptionkey@0"
I20260928 15:57:39.363255 29539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "d3db5ab25c94440a80c3ab45878cd12a"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "4b69345ff1176f017e3d2223bf5abdc6"
server_key_iv: "aca90b6c46218393742c79d037d7ef71"
server_key_version: "encryptionkey@0"
I20260928 15:57:39.368346 29539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260928 15:57:39.371555 29554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:39.372824 29539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:39.372999 29539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "d3db5ab25c94440a80c3ab45878cd12a"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "4b69345ff1176f017e3d2223bf5abdc6"
server_key_iv: "aca90b6c46218393742c79d037d7ef71"
server_key_version: "encryptionkey@0"
I20260928 15:57:39.373193 29539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:39.400017 29539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:39.403949 29539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:39.404281 29539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:39.405216 29539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:39.406543 29539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:39.406616 29539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:39.406791 29539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:39.406844 29539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:39.439826 29539 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:44653
I20260928 15:57:39.439869 29669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:44653 every 8 connection(s)
I20260928 15:57:39.441097 29539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:39.451491 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29539
I20260928 15:57:39.451705 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:39.452299 23031 external_mini_cluster.cc:1428] Setting key 61431e75db3d452b54170809957097ec
Sep 28 15:57:39 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:39.461419 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:39.443494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:51339 (local address 127.22.125.254:37163)
0928 15:57:39.443703 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:39.443710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:39.444934 (+ 1224us) server_negotiation.cc:407] Connection header received
0928 15:57:39.446772 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:39.446777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:39.446848 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:39.446935 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:39.448436 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.449167 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.450589 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.450998 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.451102 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:39.455369 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:39.455374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:39.455377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:39.455408 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:39.458435 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.459105 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.459111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.459115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.459263 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.459677 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.459683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.459686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.460015 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:39.460193 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:39.460813 (+ 620us) server_negotiation.cc:299] Negotiation successful
0928 15:57:39.461047 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 15:57:39.462235 29672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:39.444009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:51339)
0928 15:57:39.444738 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:39.444782 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:39.446533 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:39.447128 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:39.447136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:39.447641 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:39.448278 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:39.448297 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.449285 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.449288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:39.450461 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:39.450476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.451235 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.451244 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:39.451538 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:39.452312 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:39.452346 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:39.455149 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:39.458572 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:39.458578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:39.458588 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:39.458974 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:39.459386 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:39.459389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:39.459391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:39.459553 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:39.460157 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:39.460162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:39.460453 (+ 291us) client_negotiation.cc:769] Sending connection context
0928 15:57:39.460659 (+ 206us) client_negotiation.cc:240] Negotiation successful
0928 15:57:39.461055 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":260,"threads_started":1}
I20260928 15:57:39.463577 29670 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37163
I20260928 15:57:39.463825 29670 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:39.464531 29670 heartbeater.cc:511] Master 127.22.125.254:37163 requested a full tablet report, sending...
I20260928 15:57:39.465772 29205 ts_manager.cc:194] Registered new tserver with Master: d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:39.466799 29205 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:51339
I20260928 15:57:39.467820 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.253@KRBTEST.COM" created.
I20260928 15:57:39.495158 29396 heartbeater.cc:503] Master 127.22.125.254:37163 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.253.keytab.
Entry for principal oryx/127.22.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.253.keytab.
I20260928 15:57:39.512723 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:38453
--webserver_interface=127.22.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.253.keytab
--principal=oryx/127.22.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:37163,127.22.125.253:38453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:39.831115 29675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:39.831375 29675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:39.831460 29675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:39.838572 29675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:39.838668 29675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:39.838729 29675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:39.838786 29675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:39.838850 29675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:39.848223 29675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:37163,127.22.125.253:38453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.253.keytab
--principal=oryx/127.22.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:38453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
W20260928 15:57:39.848511 29675 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.125.254:37163,127.22.125.253:38453'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 15:57:39.849556 29675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:39.850770 29675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:39.862589 29681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:39.862551 29680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:39.863072 29683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:39.868772 29675 server_base.cc:1034] running on GCE node
I20260928 15:57:39.869599 29675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:39.870980 29675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:39.872219 29675 hybrid_clock.cc:648] HybridClock initialized: now 1790611059872162 us; error 36 us; skew 500 ppm
I20260928 15:57:39.872543 29675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:39 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:39.875533 29675 init.cc:377] Logged in from keytab as oryx/127.22.125.253@KRBTEST.COM (short username oryx)
I20260928 15:57:39.877185 29675 webserver.cc:460] Webserver started at http://127.22.125.253:40089/ using document root <none> and password file <none>
I20260928 15:57:39.878103 29675 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:39.878232 29675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:39.878548 29675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:39.881548 29675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/instance:
uuid: "03e6b21e733144fd9d68a8309e7361d3"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "e7952f0578c4d0aed271e2b33490dd64"
server_key_iv: "81d1f85ce5bec3427d247a2c532938a6"
server_key_version: "encryptionkey@0"
I20260928 15:57:39.882330 29675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal/instance:
uuid: "03e6b21e733144fd9d68a8309e7361d3"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "e7952f0578c4d0aed271e2b33490dd64"
server_key_iv: "81d1f85ce5bec3427d247a2c532938a6"
server_key_version: "encryptionkey@0"
I20260928 15:57:39.887229 29675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260928 15:57:39.890460 29690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:39.891690 29675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260928 15:57:39.891866 29675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
uuid: "03e6b21e733144fd9d68a8309e7361d3"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "e7952f0578c4d0aed271e2b33490dd64"
server_key_iv: "81d1f85ce5bec3427d247a2c532938a6"
server_key_version: "encryptionkey@0"
I20260928 15:57:39.892094 29675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:39.920303 29675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:39.924003 29675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:39.924281 29675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:39.942555 29675 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.253:38453
I20260928 15:57:39.942642 29743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.253:38453 every 8 connection(s)
I20260928 15:57:39.943929 29675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
I20260928 15:57:39.947738 29744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:39.951035 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29675
I20260928 15:57:39.954777 29744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 15:57:39 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.253@KRBTEST.COM
Sep 28 15:57:39 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.253@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:39.973285 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:39.957650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44566 (local address 127.22.125.254:37163)
0928 15:57:39.957871 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:39.957879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:39.958591 (+ 712us) server_negotiation.cc:407] Connection header received
0928 15:57:39.958766 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:39.958770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:39.958832 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:39.958972 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:39.960276 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.961402 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.963355 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.963768 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.963923 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:39.967459 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:39.967464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:39.967467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:39.967518 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:39.970571 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.971347 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.971352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.971354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.971479 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.971876 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.971880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.971883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.972327 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:39.972476 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:39.972917 (+ 441us) server_negotiation.cc:299] Negotiation successful
0928 15:57:39.973069 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 15:57:39.974201 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:39.957679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:44566)
0928 15:57:39.958462 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:39.958502 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:39.958651 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:39.959051 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:39.959058 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:39.959846 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:39.960130 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:39.960139 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.961520 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.961525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:39.963156 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:39.963171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.963999 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.964006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:39.964179 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:39.964767 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:39.964786 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:39.967280 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:39.970681 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:39.970693 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:39.970702 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:39.971234 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:39.971587 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:39.971592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:39.971596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:39.971761 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:39.972453 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:39.972460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:39.972627 (+ 167us) client_negotiation.cc:769] Sending connection context
0928 15:57:39.972838 (+ 211us) client_negotiation.cc:240] Negotiation successful
0928 15:57:39.973119 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":377,"threads_started":1}
I20260928 15:57:39.974220 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:39.953036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.22.125.253:38453)
0928 15:57:39.953724 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:39.953735 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:39.953771 (+ 36us) server_negotiation.cc:407] Connection header received
0928 15:57:39.955372 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:39.955393 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:39.955745 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:39.956128 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:39.957596 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.959598 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.961297 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.962329 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.962497 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:39.966823 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:39.966836 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:39.966855 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:39.966916 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:39.969686 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.970472 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.970478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.970483 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.970642 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.971040 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.971047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.971050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.971680 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:39.971972 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:39.972416 (+ 444us) server_negotiation.cc:299] Negotiation successful
0928 15:57:39.972816 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":319,"threads_started":1}
I20260928 15:57:39.975102 29534 heartbeater.cc:503] Master 127.22.125.254:37163 was elected leader, sending a full tablet report...
I20260928 15:57:39.977048 29744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 15:57:39 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611059, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.253@KRBTEST.COM for oryx/127.22.125.253@KRBTEST.COM
I20260928 15:57:39.988524 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:39.977720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.0.0.1:54392)
0928 15:57:39.977902 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:39.977919 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:39.978023 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:39.978302 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:39.978306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:39.978641 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:39.978755 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:39.978762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.979687 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.979692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:39.980620 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:39.980631 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.981328 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.981333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:39.981551 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:39.982006 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:39.982021 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:39.983805 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:39.986578 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:39.986582 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:39.986584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:39.986930 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:39.987217 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:39.987220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:39.987222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:39.987336 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:39.987785 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:39.987788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:39.987911 (+ 123us) client_negotiation.cc:769] Sending connection context
0928 15:57:39.988015 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 15:57:39.988269 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":1}
I20260928 15:57:39.988593 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:39.977783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54392 (local address 127.22.125.253:38453)
0928 15:57:39.977922 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:39.977928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:39.977968 (+ 40us) server_negotiation.cc:407] Connection header received
0928 15:57:39.978134 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:39.978138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:39.978198 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:39.978286 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:39.978890 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.979553 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.980754 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:39.981192 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:39.981322 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:39.983964 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:39.983969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:39.983971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:39.983998 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:39.986463 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.987024 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.987027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.987029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.987129 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:39.987426 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:39.987429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:39.987431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:39.987692 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:39.987794 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:39.988158 (+ 364us) server_negotiation.cc:299] Negotiation successful
0928 15:57:39.988371 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":51}
I20260928 15:57:39.992728 29744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Bootstrap starting.
I20260928 15:57:39.995694 29744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:39.996838 29744 log.cc:824] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:39.999169 29744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: No bootstrap required, opened a new log
I20260928 15:57:40.005941 29744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } }
I20260928 15:57:40.006237 29744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:40.006307 29744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e6b21e733144fd9d68a8309e7361d3, State: Initialized, Role: FOLLOWER
I20260928 15:57:40.006771 29744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } }
I20260928 15:57:40.007483 29750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } } }
I20260928 15:57:40.007820 29750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:40.008463 29744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 15:57:40.017380 29761 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 15:57:40.017552 29761 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 15:57:40.020238 29675 master_runner.cc:428] Detected that this master 03e6b21e733144fd9d68a8309e7361d3 is joining an existing cluster
I20260928 15:57:40.020331 29675 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.125.253:38453
I20260928 15:57:40.038106 29750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:40.038443 29750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } }
I20260928 15:57:40.040007 29750 leader_election.cc:288] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163)
I20260928 15:57:40.040895 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03e6b21e733144fd9d68a8309e7361d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6991a5163c3e404fb77e798a3f36fee3" is_pre_election: true
I20260928 15:57:40.041201 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.025572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54406 (local address 127.22.125.253:38453)
0928 15:57:40.025754 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.025759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.025812 (+ 53us) server_negotiation.cc:407] Connection header received
0928 15:57:40.026057 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.026062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.026133 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.026232 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.027210 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.028196 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.030667 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.031385 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.031479 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.034196 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.034202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.034208 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.034250 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.038009 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.038846 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.038867 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.038871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.039030 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.039441 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.039446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.039452 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.039830 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.039999 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.040577 (+ 578us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.040809 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 15:57:40.042018 29692 leader_election.cc:302] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 03e6b21e733144fd9d68a8309e7361d3; no voters: 6991a5163c3e404fb77e798a3f36fee3
I20260928 15:57:40.042198 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.025105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.0.0.1:54406)
0928 15:57:40.025728 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.025749 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.025893 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.026244 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.026248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.026812 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.027075 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.027084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.028416 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.028420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.030505 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.030522 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.031778 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.031785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.031970 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.032697 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.032722 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.034005 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.038199 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.038204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.038207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.038725 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.039161 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.039165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.039169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.039329 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.040138 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.040142 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.040310 (+ 168us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.041421 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.041696 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":37,"thread_start_us":647,"threads_started":1}
I20260928 15:57:40.042713 29750 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260928 15:57:40.047106 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.024284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:44572)
0928 15:57:40.024938 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.024959 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.025089 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.025631 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.025636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.026486 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.026916 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.026931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.028508 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.028517 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.030785 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.030803 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.032227 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.032237 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.032438 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.033033 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.033063 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.034356 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.043782 (+ 9426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.043789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.043796 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.044332 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.044803 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.044810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.044816 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.044998 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.045768 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.045775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.045998 (+ 223us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.046170 (+ 172us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.046551 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":1,"thread_start_us":360,"threads_started":1}
I20260928 15:57:40.048157 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.024276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.22.125.254:37163)
0928 15:57:40.024476 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.024484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.025045 (+ 561us) server_negotiation.cc:407] Connection header received
0928 15:57:40.025230 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.025236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.025319 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.025431 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.027109 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.028309 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.030961 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.031752 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.031860 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.034590 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.034598 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.034605 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.034652 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.043628 (+ 8976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.044497 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.044505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.044512 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.044670 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.045137 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.045145 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.045151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.045546 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.045702 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.047424 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.047686 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
WARNING: no policy specified for oryx/127.22.125.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.252@KRBTEST.COM" created.
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.254@KRBTEST.COM for oryx/127.22.125.253@KRBTEST.COM
I20260928 15:57:40.076459 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.054952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.0.0.1:54410)
0928 15:57:40.055630 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.055683 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.060212 (+ 4529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.060653 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.060660 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.061304 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.061516 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.061525 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.062654 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.062662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.063929 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.063948 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.065005 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.065011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.065226 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.065969 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.065995 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.069042 (+ 3047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.073256 (+ 4214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.073273 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.073283 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.073922 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.074404 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.074411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.074416 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.074592 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.075249 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.075261 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.075484 (+ 223us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.075691 (+ 207us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.076074 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":286,"threads_started":1}
I20260928 15:57:40.076501 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.054838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.22.125.253:38453)
0928 15:57:40.055028 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.055037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.055790 (+ 753us) server_negotiation.cc:407] Connection header received
0928 15:57:40.060424 (+ 4634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.060428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.060493 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.060605 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.061659 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.062529 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.064142 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.064824 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.064967 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.069243 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.069249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.069255 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.069302 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.073113 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.074096 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.074102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.074106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.074264 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.074730 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.074735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.074739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.075087 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.075239 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.075833 (+ 594us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.076116 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 15:57:40.078776 29205 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.125.253:38453: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3"
member_type: NON_VOTER
last_known_addr {
host: "127.22.125.253"
port: 38453
}
attrs {
promote: true
}
}
dest_uuid: "6991a5163c3e404fb77e798a3f36fee3"
cas_config_opid_index: -1
I20260928 15:57:40.080307 29205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.252.keytab.
Entry for principal oryx/127.22.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.252.keytab.
I20260928 15:57:40.084399 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:45153
--webserver_interface=127.22.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.252.keytab
--principal=oryx/127.22.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 15:57:40.096436 29717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:40.101616 29717 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Refusing update from remote peer 6991a5163c3e404fb77e798a3f36fee3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 15:57:40.102053 29773 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.106880 29774 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:40.124300 29773 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.125.253:38453
I20260928 15:57:40.126003 29774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.126272 29675 master.cc:537] Master@127.22.125.253:38453 shutting down...
I20260928 15:57:40.126343 29774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:40.144627 29717 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.173592 29785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.174005 29785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: LEARNER
I20260928 15:57:40.174078 29750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6991a5163c3e404fb77e798a3f36fee3. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.174319 29750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: LEARNER
I20260928 15:57:40.174499 29779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.174860 29779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: LEARNER
I20260928 15:57:40.178329 29675 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 LEARNER]: Raft consensus shutting down.
I20260928 15:57:40.178836 29675 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 LEARNER]: Raft consensus is shut down!
I20260928 15:57:40.179003 29675 tablet_replica.cc:330] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: stopping tablet replica
I20260928 15:57:40.197700 29675 master.cc:552] Master@127.22.125.253:38453 shutdown complete.
I20260928 15:57:40.197851 29675 master_runner.cc:305] Clearing existing system tablet
I20260928 15:57:40.198323 29675 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:40.198427 29675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:40.201598 29675 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.005s sys 0.000s
I20260928 15:57:40.203275 29787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:40.203657 29675 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:40.203918 29675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260928 15:57:40.203997 29675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
uuid: "03e6b21e733144fd9d68a8309e7361d3"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "e7952f0578c4d0aed271e2b33490dd64"
server_key_iv: "81d1f85ce5bec3427d247a2c532938a6"
server_key_version: "encryptionkey@0"
I20260928 15:57:40.209070 29675 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:40.216007 29675 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 15:57:40.216245 29675 log.cc:1196] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 15:57:40.216781 29675 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Deleting consensus metadata
I20260928 15:57:40.217034 29675 master_runner.cc:315] Copying system tablet from 127.22.125.254:37163
I20260928 15:57:40.221633 29675 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: tablet copy: Beginning tablet copy session from remote peer at address 127.22.125.254:37163
I20260928 15:57:40.238408 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.223562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44586 (local address 127.22.125.254:37163)
0928 15:57:40.223756 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.223763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.224338 (+ 575us) server_negotiation.cc:407] Connection header received
0928 15:57:40.224532 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.224537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.224638 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.224754 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.225826 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.227093 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.228996 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.229632 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.229760 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.231983 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.231989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.231993 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.232030 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.235538 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.236328 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.236334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.236337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.236489 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.236928 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.236933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.236936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.237281 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.237426 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.237878 (+ 452us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.238095 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 15:57:40.238492 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.223482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:44586)
0928 15:57:40.224203 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.224227 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.224391 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.224807 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.224812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.225334 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.225637 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.225649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.227248 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.227255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.228834 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.228849 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.229801 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.229807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.230010 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.230733 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.230756 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.231778 (+ 1022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.235689 (+ 3911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.235695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.235699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.236209 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.236612 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.236618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.236622 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.236795 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.237411 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.237416 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.237590 (+ 174us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.237751 (+ 161us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.238048 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":392,"threads_started":1}
I20260928 15:57:40.239655 29226 tablet_copy_service.cc:140] P 6991a5163c3e404fb77e798a3f36fee3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 03e6b21e733144fd9d68a8309e7361d3 ({username='oryx', principal='oryx/127.22.125.253@KRBTEST.COM'} at 127.0.0.1:44586)
I20260928 15:57:40.239924 29226 tablet_copy_service.cc:161] P 6991a5163c3e404fb77e798a3f36fee3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 03e6b21e733144fd9d68a8309e7361d3 at {username='oryx', principal='oryx/127.22.125.253@KRBTEST.COM'} at 127.0.0.1:44586: session id = 03e6b21e733144fd9d68a8309e7361d3-00000000000000000000000000000000
I20260928 15:57:40.244546 29226 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 15:57:40.248178 29675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 15:57:40.254969 29675 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: tablet copy: Starting download of 0 data blocks...
I20260928 15:57:40.255331 29675 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: tablet copy: Starting download of 1 WAL segments...
I20260928 15:57:40.259752 29675 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 15:57:40.264209 29226 tablet_copy_service.cc:342] P 6991a5163c3e404fb77e798a3f36fee3: Request end of tablet copy session 03e6b21e733144fd9d68a8309e7361d3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.125.253@KRBTEST.COM'} at 127.0.0.1:44586
I20260928 15:57:40.264403 29226 tablet_copy_service.cc:433] P 6991a5163c3e404fb77e798a3f36fee3: ending tablet copy session 03e6b21e733144fd9d68a8309e7361d3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 03e6b21e733144fd9d68a8309e7361d3
I20260928 15:57:40.278671 29675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:40.281335 29794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:40.281651 29795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:40.283269 29797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:40.283687 29675 server_base.cc:1034] running on GCE node
I20260928 15:57:40.284103 29675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:40.284795 29675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:40.286092 29675 hybrid_clock.cc:648] HybridClock initialized: now 1790611060286022 us; error 34 us; skew 500 ppm
I20260928 15:57:40.286408 29675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611060, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:40.289047 29675 init.cc:377] Logged in from keytab as oryx/127.22.125.253@KRBTEST.COM (short username oryx)
I20260928 15:57:40.290594 29675 webserver.cc:460] Webserver started at http://127.22.125.253:43323/ using document root <none> and password file <none>
I20260928 15:57:40.291106 29675 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:40.291275 29675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:40.294953 29675 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260928 15:57:40.297171 29804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:40.297557 29675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 15:57:40.297739 29675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
uuid: "03e6b21e733144fd9d68a8309e7361d3"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "e7952f0578c4d0aed271e2b33490dd64"
server_key_iv: "81d1f85ce5bec3427d247a2c532938a6"
server_key_version: "encryptionkey@0"
I20260928 15:57:40.297974 29675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:40.313311 29675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:40.317530 29675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:40.347664 29675 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.253:38453
I20260928 15:57:40.347698 29856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.253:38453 every 8 connection(s)
I20260928 15:57:40.348956 29675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
I20260928 15:57:40.354810 29857 sys_catalog.cc:263] Verifying existing consensus state
I20260928 15:57:40.358654 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Bootstrap starting.
I20260928 15:57:40.375149 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 15:57:40.375802 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: Bootstrap complete.
I20260928 15:57:40.376969 29857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } }
I20260928 15:57:40.377234 29857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 03e6b21e733144fd9d68a8309e7361d3, State: Initialized, Role: LEARNER
I20260928 15:57:40.377573 29857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } }
I20260928 15:57:40.378464 29860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.378751 29857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:40.378918 29860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: LEARNER
I20260928 15:57:40.395767 29871 catalog_manager.cc:1259] Loaded cluster ID: 6aa39b3fca66401e9cb6881c23cd4302
I20260928 15:57:40.395897 29871 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: loading cluster ID for follower catalog manager: success
I20260928 15:57:40.398582 29871 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: acquiring CA information for follower catalog manager: success
I20260928 15:57:40.400699 29871 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 15:57:40.469599 29670 heartbeater.cc:503] Master 127.22.125.254:37163 was elected leader, sending a full tablet report...
W20260928 15:57:40.510474 29775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:40.510812 29775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:40.510982 29775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:40.518497 29775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:40.518600 29775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:40.518695 29775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:40.518800 29775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:40.518865 29775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:40.533919 29775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.252.keytab
--principal=oryx/127.22.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:45153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:40.535277 29775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:40.536649 29775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:40.548540 29876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:40.548553 29877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:40.548697 29879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:40.549396 29775 server_base.cc:1034] running on GCE node
I20260928 15:57:40.550108 29775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:40.551427 29775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:40.552664 29775 hybrid_clock.cc:648] HybridClock initialized: now 1790611060552624 us; error 51 us; skew 500 ppm
I20260928 15:57:40.553009 29775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611060, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:40.555966 29775 init.cc:377] Logged in from keytab as oryx/127.22.125.252@KRBTEST.COM (short username oryx)
I20260928 15:57:40.557302 29775 webserver.cc:460] Webserver started at http://127.22.125.252:42473/ using document root <none> and password file <none>
I20260928 15:57:40.557945 29775 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:40.558079 29775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:40.558378 29775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:40.561319 29775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/instance:
uuid: "cebe265c0fea44a194e527f18e02d7d5"
format_stamp: "Formatted at 2026-09-28 15:57:40 on dist-test-slave-9cqd"
server_key: "1b147182320acad82c88345ce17d2e79"
server_key_iv: "50b86c447644fcf85eec2ac578edd4a3"
server_key_version: "encryptionkey@0"
I20260928 15:57:40.562116 29775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal/instance:
uuid: "cebe265c0fea44a194e527f18e02d7d5"
format_stamp: "Formatted at 2026-09-28 15:57:40 on dist-test-slave-9cqd"
server_key: "1b147182320acad82c88345ce17d2e79"
server_key_iv: "50b86c447644fcf85eec2ac578edd4a3"
server_key_version: "encryptionkey@0"
I20260928 15:57:40.566929 29775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260928 15:57:40.570098 29887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:40.571352 29775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260928 15:57:40.571521 29775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
uuid: "cebe265c0fea44a194e527f18e02d7d5"
format_stamp: "Formatted at 2026-09-28 15:57:40 on dist-test-slave-9cqd"
server_key: "1b147182320acad82c88345ce17d2e79"
server_key_iv: "50b86c447644fcf85eec2ac578edd4a3"
server_key_version: "encryptionkey@0"
I20260928 15:57:40.571697 29775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:40.603592 29775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:40.607376 29775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:40.607663 29775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:40.626238 29775 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.252:45153
I20260928 15:57:40.626258 29939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.252:45153 every 8 connection(s)
I20260928 15:57:40.627643 29775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
I20260928 15:57:40.631204 29940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:40.633898 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29775
I20260928 15:57:40.638831 29940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.252@KRBTEST.COM
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611060, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.252@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:40.656806 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.641926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44602 (local address 127.22.125.254:37163)
0928 15:57:40.642117 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.642123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.643224 (+ 1101us) server_negotiation.cc:407] Connection header received
0928 15:57:40.643441 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.643451 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.643516 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.643626 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.644960 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.645939 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.647369 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.648101 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.648244 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.651421 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.651425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.651429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.651464 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.653950 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.654675 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.654682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.654685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.654813 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.655276 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.655283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.655289 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.655553 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.655706 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.656233 (+ 527us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.656460 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 15:57:40.657321 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.635874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.22.125.252:45153)
0928 15:57:40.636634 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.636645 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.636691 (+ 46us) server_negotiation.cc:407] Connection header received
0928 15:57:40.638412 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.638432 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.638857 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.639235 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.641027 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.642989 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.644364 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.645266 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.645403 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.650089 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.650101 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.650119 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.650160 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.653249 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.653955 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.653961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.653967 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.654114 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.654518 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.654522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.654524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.654999 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.655352 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.655716 (+ 364us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.656183 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":371,"threads_started":1}
I20260928 15:57:40.657385 29944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.642019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:44602)
0928 15:57:40.643106 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.643150 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.643286 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.643692 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.643699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.644460 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.644794 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.644806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.646087 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.646094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.647214 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.647225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.648297 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.648302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.648422 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.649015 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.649036 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.651214 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.654084 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.654095 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.654102 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.654557 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.654940 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.654945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.654948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.655157 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.655683 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.655690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.655860 (+ 170us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.656109 (+ 249us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.656351 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"spinlock_wait_cycles":28928,"thread_start_us":626,"threads_started":1}
I20260928 15:57:40.658836 29940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611060, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.252@KRBTEST.COM for oryx/127.22.125.253@KRBTEST.COM
I20260928 15:57:40.671525 29944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.659392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.0.0.1:54418)
0928 15:57:40.659551 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.659569 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.659649 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.660530 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.660535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.660874 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.660995 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.661003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.662276 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.662284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.663203 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.663214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.663916 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.663921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.664166 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.664702 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.664717 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.666512 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.669253 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.669261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.669265 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.669624 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.669989 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.669994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.669998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.670130 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.670616 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.670620 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.670768 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.670907 (+ 139us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.671177 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 15:57:40.671548 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.659609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.22.125.253:38453)
0928 15:57:40.660260 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.660264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.660280 (+ 16us) server_negotiation.cc:407] Connection header received
0928 15:57:40.660327 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.660330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.660396 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.660512 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.661147 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.662151 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.663348 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.663749 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.663918 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.666717 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.666722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.666725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.666755 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.669126 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.669740 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.669747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.669750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.669879 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.670238 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.670244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.670247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.670500 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.670664 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.671003 (+ 339us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.671197 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":329,"threads_started":1}
I20260928 15:57:40.672627 29940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611060, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.252@KRBTEST.COM for oryx/127.22.125.252@KRBTEST.COM
I20260928 15:57:40.683584 29946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.683969 29946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:40.685307 29944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.673389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.0.0.1:43846)
0928 15:57:40.673561 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.673585 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.673666 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.673910 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.673913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.674199 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.674334 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.674341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.675255 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.675259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.676416 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.676427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.677100 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.677104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.677255 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.677710 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.677725 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.679592 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.682414 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.682419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.682422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.682813 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.683150 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.683156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.683162 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.683349 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.684075 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.684079 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.684220 (+ 141us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.684413 (+ 193us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.684743 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260928 15:57:40.686357 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.673377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.22.125.252:45153)
0928 15:57:40.673521 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.673525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.673643 (+ 118us) server_negotiation.cc:407] Connection header received
0928 15:57:40.673744 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.673747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.673797 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.673872 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.674441 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.675146 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.676538 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.676966 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.677077 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.679761 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.679765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.679768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.679793 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.682247 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.682931 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.682938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.682942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.683049 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.683489 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.683496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.683502 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.683859 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.684001 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.685629 (+ 1628us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.685910 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 15:57:40.692246 29940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Bootstrap starting.
I20260928 15:57:40.694785 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.683686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.0.0.1:54426)
0928 15:57:40.688696 (+ 5010us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.688716 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.688916 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.689474 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.689480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.689506 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:40.689775 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.689787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.691218 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.691223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.692650 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.692663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.694225 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.694229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.694366 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.694410 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.694474 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.694564 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4870,"thread_start_us":7425,"threads_started":1}
I20260928 15:57:40.694870 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.684144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.22.125.253:38453)
0928 15:57:40.688277 (+ 4133us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.688283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.688795 (+ 512us) server_negotiation.cc:407] Connection header received
0928 15:57:40.689067 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.689072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.689183 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.689263 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:40.689974 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.691090 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.692801 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.694057 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.694252 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.694455 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.694509 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.694632 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3986}
I20260928 15:57:40.695993 29860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6991a5163c3e404fb77e798a3f36fee3. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.696297 29860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: LEARNER
I20260928 15:57:40.696405 29940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:40.697034 29947 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: attempting to promote NON_VOTER 03e6b21e733144fd9d68a8309e7361d3 to VOTER
I20260928 15:57:40.697510 29940 log.cc:824] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:40.697873 29947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } }
I20260928 15:57:40.699600 29831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 LEARNER]: Refusing update from remote peer 6991a5163c3e404fb77e798a3f36fee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 15:57:40.700202 29948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 15:57:40.700525 29940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: No bootstrap required, opened a new log
I20260928 15:57:40.703150 29947 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } }
I20260928 15:57:40.703830 29831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } }
I20260928 15:57:40.706804 29947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } }
I20260928 15:57:40.707026 29947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:40.707193 29946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } }
I20260928 15:57:40.707389 29946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:40.707671 29860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } }
I20260928 15:57:40.707825 29860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:40.708652 29940 raft_consensus.cc:348] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } }
I20260928 15:57:40.708974 29940 raft_consensus.cc:374] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:40.709110 29940 raft_consensus.cc:729] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebe265c0fea44a194e527f18e02d7d5, State: Initialized, Role: FOLLOWER
I20260928 15:57:40.709728 29940 consensus_queue.cc:260] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } }
I20260928 15:57:40.710623 29953 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } } }
I20260928 15:57:40.711105 29953 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:40.711894 29940 sys_catalog.cc:564] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 15:57:40.720312 29964 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 15:57:40.720468 29964 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 15:57:40.724151 29775 master_runner.cc:428] Detected that this master cebe265c0fea44a194e527f18e02d7d5 is joining an existing cluster
I20260928 15:57:40.724285 29775 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.125.252:45153
I20260928 15:57:40.746865 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.728277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.22.125.254:37163)
0928 15:57:40.728479 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.728486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.729216 (+ 730us) server_negotiation.cc:407] Connection header received
0928 15:57:40.729494 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.729499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.729585 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.729666 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.731168 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.732345 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.734515 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.735193 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.735293 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.738549 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.738556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.738561 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.738600 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.743175 (+ 4575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.743992 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.743997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.744001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.744178 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.744687 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.744693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.744697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.745047 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.745168 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.745855 (+ 687us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.746089 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 15:57:40.747993 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.728373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:44606)
0928 15:57:40.729097 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.729130 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.729330 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.729888 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.729892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.730596 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.730993 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.731007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.732524 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.732532 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.734349 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.734367 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.735582 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.735589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.735795 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.736772 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.736796 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.738294 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.743355 (+ 5061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.743360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.743363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.743836 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.744332 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.744337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.744340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.744539 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.745268 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.745273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.745497 (+ 224us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.747139 (+ 1642us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.747458 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":664,"threads_started":1}
I20260928 15:57:40.758662 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.728815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.22.125.252:45153)
0928 15:57:40.729030 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.729036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.740178 (+ 11142us) server_negotiation.cc:407] Connection header received
0928 15:57:40.740594 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.740604 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.740688 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.740765 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.742547 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.744116 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.747303 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.747802 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.747873 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.750489 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.750494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.750498 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.750537 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.754346 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.756303 (+ 1957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.756308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.756312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.756472 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.756951 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.756957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.756960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.757340 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.757491 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.758065 (+ 574us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.758296 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"spinlock_wait_cycles":26880}
I20260928 15:57:40.760231 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.729025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.0.0.1:43862)
0928 15:57:40.740061 (+ 11036us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.740088 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.740438 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.740973 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.740980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.741885 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.742336 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.742351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.744402 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.744409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.746548 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.746567 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.748182 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.748190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.748371 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.749026 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.749066 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.750313 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.755621 (+ 5308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.755627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.755631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.756168 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.756624 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.756636 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.756640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.756816 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.757556 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.757563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.757715 (+ 152us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.759410 (+ 1695us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.759707 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10843,"thread_start_us":17797,"threads_started":1}
I20260928 15:57:40.766724 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.730092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.22.125.253:38453)
0928 15:57:40.730306 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.730314 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.747208 (+ 16894us) server_negotiation.cc:407] Connection header received
0928 15:57:40.749356 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.749362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.749454 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.749541 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.751022 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.752206 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.754628 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.755270 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.755412 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.758369 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.758379 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.758386 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.758437 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.763479 (+ 5042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.764340 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.764346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.764350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.764504 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.764976 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.764982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.764985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.765353 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.765501 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.766147 (+ 646us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.766374 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 15:57:40.767690 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.729577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.0.0.1:54434)
0928 15:57:40.747112 (+ 17535us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.747133 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.748411 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.749758 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.749763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.750483 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.750828 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.750842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.752405 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.752413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.754358 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.754368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.755521 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.755530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.755840 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.756601 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.756633 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.758080 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.763633 (+ 5553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.763638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.763641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.764194 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.764656 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.764661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.764664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.764840 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.765633 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.765637 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.765833 (+ 196us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.766911 (+ 1078us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.767220 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17400,"spinlock_wait_cycles":5504,"thread_start_us":10611,"threads_started":1}
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.254@KRBTEST.COM for oryx/127.22.125.252@KRBTEST.COM
I20260928 15:57:40.773579 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29267
I20260928 15:57:40.777794 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.754729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.0.0.1:43866)
0928 15:57:40.755037 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.755061 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.755188 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.759490 (+ 4302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.759498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.760155 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.760393 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.760407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.762198 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.762207 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.764135 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.764156 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.765834 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.765844 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.766098 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.766955 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.766977 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.770524 (+ 3547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.774873 (+ 4349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.774878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.774881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.775462 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.775903 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.775908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.775912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.776124 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.776802 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.776807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.776990 (+ 183us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.777150 (+ 160us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.777438 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260928 15:57:40.778173 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.754799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.22.125.252:45153)
0928 15:57:40.758901 (+ 4102us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.758908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.758925 (+ 17us) server_negotiation.cc:407] Connection header received
0928 15:57:40.759014 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.759019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.759105 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.759227 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.760735 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.762049 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.764426 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.765427 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.765571 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.770734 (+ 5163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.770744 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.770755 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.770803 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.774718 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.775628 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.775634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.775638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.775770 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.776272 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.776277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.776282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.776656 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.776810 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.777421 (+ 611us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.777652 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4009,"thread_start_us":17465,"threads_started":1}
I20260928 15:57:40.779673 29205 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.125.252:45153: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5"
member_type: NON_VOTER
last_known_addr {
host: "127.22.125.252"
port: 45153
}
attrs {
promote: true
}
}
dest_uuid: "6991a5163c3e404fb77e798a3f36fee3"
cas_config_opid_index: 6
I20260928 15:57:40.780530 29205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } }
I20260928 15:57:40.783113 29831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Refusing update from remote peer 6991a5163c3e404fb77e798a3f36fee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 15:57:40.783201 29914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:40.786553 29914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Refusing update from remote peer 6991a5163c3e404fb77e798a3f36fee3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 15:57:40.786840 29946 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260928 15:57:40.787339 29948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.193@KRBTEST.COM
I20260928 15:57:40.789963 29948 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:40.790992 29831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:40.799067 29974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.775037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.22.125.193:42437)
0928 15:57:40.775713 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.775722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.775750 (+ 28us) server_negotiation.cc:407] Connection header received
0928 15:57:40.775818 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.775831 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.776309 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.778724 (+ 2415us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.779003 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.780564 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.783594 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.784538 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.785261 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.790541 (+ 5280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.790556 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.790575 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.790620 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.794687 (+ 4067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.795632 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.795641 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.795644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.795822 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.796367 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.796372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.796375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.796820 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.798147 (+ 1327us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.798308 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.798574 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":327,"threads_started":1}
I20260928 15:57:40.798942 29860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:40.799302 29860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:40.800761 29391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:40.804948 29948 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.125.252:45153
I20260928 15:57:40.805621 29775 master.cc:537] Master@127.22.125.252:45153 shutting down...
I20260928 15:57:40.806723 29948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:40.806990 29948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:40.807605 29947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:40.808012 29947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:40.810904 29913 raft_consensus.cc:858] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } }
W20260928 15:57:40.818101 29913 raft_consensus.cc:1565] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } }
I20260928 15:57:40.844539 29913 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.849129 29981 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.849593 29981 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260928 15:57:40.849619 29978 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.849997 29978 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260928 15:57:40.850026 29953 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6991a5163c3e404fb77e798a3f36fee3. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: NON_VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: true } } }
I20260928 15:57:40.850219 29953 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260928 15:57:40.851677 29948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:40.851979 29948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
W20260928 15:57:40.852670 29192 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260928 15:57:40.854060 29775 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260928 15:57:40.854529 29775 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
W20260928 15:57:40.854485 29192 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 -> Peer cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252:45153): Couldn't send request to peer cebe265c0fea44a194e527f18e02d7d5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 15:57:40.854703 29775 tablet_replica.cc:330] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: stopping tablet replica
I20260928 15:57:40.863117 29775 master.cc:552] Master@127.22.125.252:45153 shutdown complete.
I20260928 15:57:40.863209 29775 master_runner.cc:305] Clearing existing system tablet
I20260928 15:57:40.863610 29775 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:40.863725 29775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:40.867254 29775 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260928 15:57:40.868911 29984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:40.869248 29775 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:40.869669 29775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 15:57:40.869817 29775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
uuid: "cebe265c0fea44a194e527f18e02d7d5"
format_stamp: "Formatted at 2026-09-28 15:57:40 on dist-test-slave-9cqd"
server_key: "1b147182320acad82c88345ce17d2e79"
server_key_iv: "50b86c447644fcf85eec2ac578edd4a3"
server_key_version: "encryptionkey@0"
I20260928 15:57:40.874500 29775 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:40.881704 29775 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 15:57:40.881935 29775 log.cc:1196] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 15:57:40.882542 29775 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Deleting consensus metadata
I20260928 15:57:40.882854 29775 master_runner.cc:315] Copying system tablet from 127.22.125.254:37163
I20260928 15:57:40.885620 29775 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: tablet copy: Beginning tablet copy session from remote peer at address 127.22.125.254:37163
I20260928 15:57:40.903187 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.886532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44618 (local address 127.22.125.254:37163)
0928 15:57:40.886747 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:40.886755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:40.887264 (+ 509us) server_negotiation.cc:407] Connection header received
0928 15:57:40.887548 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:40.887555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:40.887635 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:40.887706 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:40.889017 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.890210 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.892128 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.892841 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.892936 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.895910 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:40.895917 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:40.895922 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:40.895961 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:40.899825 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.900760 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.900767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.900771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.900953 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:40.901414 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:40.901420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:40.901425 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:40.901786 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:40.901916 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:40.902533 (+ 617us) server_negotiation.cc:299] Negotiation successful
0928 15:57:40.902771 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 15:57:40.904491 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:40.886475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:44618)
0928 15:57:40.887144 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:40.887172 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:40.887396 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:40.887906 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:40.887911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:40.888579 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:40.888845 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.888857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.890378 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.890383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.891941 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:40.891955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:40.893487 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:40.893493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:40.893692 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:40.894477 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:40.894506 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:40.895700 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:40.900094 (+ 4394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.900099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.900104 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.900621 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.901103 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:40.901108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:40.901112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:40.901290 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:40.902005 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:40.902010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:40.902236 (+ 226us) client_negotiation.cc:769] Sending connection context
0928 15:57:40.903716 (+ 1480us) client_negotiation.cc:240] Negotiation successful
0928 15:57:40.904032 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":403,"threads_started":1}
I20260928 15:57:40.905390 29226 tablet_copy_service.cc:140] P 6991a5163c3e404fb77e798a3f36fee3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cebe265c0fea44a194e527f18e02d7d5 ({username='oryx', principal='oryx/127.22.125.252@KRBTEST.COM'} at 127.0.0.1:44618)
I20260928 15:57:40.905567 29226 tablet_copy_service.cc:161] P 6991a5163c3e404fb77e798a3f36fee3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cebe265c0fea44a194e527f18e02d7d5 at {username='oryx', principal='oryx/127.22.125.252@KRBTEST.COM'} at 127.0.0.1:44618: session id = cebe265c0fea44a194e527f18e02d7d5-00000000000000000000000000000000
I20260928 15:57:40.909427 29226 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 15:57:40.911254 29775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 15:57:40.914918 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29267
I20260928 15:57:40.918329 29775 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: tablet copy: Starting download of 0 data blocks...
I20260928 15:57:40.918695 29775 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: tablet copy: Starting download of 1 WAL segments...
I20260928 15:57:40.921332 29775 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 15:57:40.925561 29226 tablet_copy_service.cc:342] P 6991a5163c3e404fb77e798a3f36fee3: Request end of tablet copy session cebe265c0fea44a194e527f18e02d7d5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.125.252@KRBTEST.COM'} at 127.0.0.1:44618
I20260928 15:57:40.925705 29226 tablet_copy_service.cc:433] P 6991a5163c3e404fb77e798a3f36fee3: ending tablet copy session cebe265c0fea44a194e527f18e02d7d5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cebe265c0fea44a194e527f18e02d7d5
I20260928 15:57:40.936142 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:42437
--local_ip_for_outbound_sockets=127.22.125.193
--tserver_master_addrs=127.22.125.254:37163
--webserver_port=45313
--webserver_interface=127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab
--principal=oryx/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 15:57:40.938915 29775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:40.941821 29991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:40.947906 29992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:40.959717 29775 server_base.cc:1034] running on GCE node
W20260928 15:57:40.959915 29995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:40.960369 29775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:40.961072 29775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:40.962267 29775 hybrid_clock.cc:648] HybridClock initialized: now 1790611060962217 us; error 44 us; skew 500 ppm
I20260928 15:57:40.962570 29775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:40 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611060, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:40.964741 29775 init.cc:377] Logged in from keytab as oryx/127.22.125.252@KRBTEST.COM (short username oryx)
I20260928 15:57:40.968959 29775 webserver.cc:460] Webserver started at http://127.22.125.252:33553/ using document root <none> and password file <none>
I20260928 15:57:40.969317 29775 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:40.969412 29775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:40.972632 29775 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 15:57:40.974599 30002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:40.975019 29775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260928 15:57:40.975234 29775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
uuid: "cebe265c0fea44a194e527f18e02d7d5"
format_stamp: "Formatted at 2026-09-28 15:57:40 on dist-test-slave-9cqd"
server_key: "1b147182320acad82c88345ce17d2e79"
server_key_iv: "50b86c447644fcf85eec2ac578edd4a3"
server_key_version: "encryptionkey@0"
I20260928 15:57:40.975513 29775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:41.002789 29775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:41.006850 29775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:41.034737 29775 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.252:45153
I20260928 15:57:41.034791 30054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.252:45153 every 8 connection(s)
I20260928 15:57:41.035988 29775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
I20260928 15:57:41.041265 30055 sys_catalog.cc:263] Verifying existing consensus state
I20260928 15:57:41.045259 30055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Bootstrap starting.
I20260928 15:57:41.060942 30055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 15:57:41.061522 30055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Bootstrap complete.
I20260928 15:57:41.062628 30055 raft_consensus.cc:348] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } }
I20260928 15:57:41.062927 30055 raft_consensus.cc:729] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cebe265c0fea44a194e527f18e02d7d5, State: Initialized, Role: LEARNER
I20260928 15:57:41.063230 30055 consensus_queue.cc:260] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } }
I20260928 15:57:41.064318 30058 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:41.064649 30055 sys_catalog.cc:564] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:41.064836 30058 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: LEARNER
I20260928 15:57:41.075664 30070 catalog_manager.cc:1259] Loaded cluster ID: 6aa39b3fca66401e9cb6881c23cd4302
I20260928 15:57:41.075763 30070 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: loading cluster ID for follower catalog manager: success
I20260928 15:57:41.079226 30070 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: acquiring CA information for follower catalog manager: success
I20260928 15:57:41.081545 30070 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 15:57:41.296398 29993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:41.296716 29993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:41.296886 29993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:41.304539 29993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:41.304693 29993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:41.304863 29993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:41.314682 29993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab
--principal=oryx/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:42437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=45313
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:41.315972 29993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:41.317301 29993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 15:57:41.320395 29946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:41.320744 29946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:41.328603 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.320933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.0.0.1:43874)
0928 15:57:41.321614 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:41.321665 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:41.321875 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:41.322727 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:41.322733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:41.322764 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:41.322991 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.323003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.324595 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.324605 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.326274 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.326287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.327868 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.327872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.328014 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.328090 (+ 76us) client_negotiation.cc:769] Sending connection context
0928 15:57:41.328177 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 15:57:41.328283 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":365,"threads_started":1}
I20260928 15:57:41.328699 30075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.321767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.22.125.252:45153)
0928 15:57:41.322399 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:41.322404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:41.322419 (+ 15us) server_negotiation.cc:407] Connection header received
0928 15:57:41.322481 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:41.322484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:41.322558 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:41.322652 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:41.323161 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.324415 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.326435 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.327700 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.327911 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.328111 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:41.328213 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 15:57:41.328369 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":357,"threads_started":1}
I20260928 15:57:41.329535 30030 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260928 15:57:41.330225 30058 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6991a5163c3e404fb77e798a3f36fee3. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: NON_VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: true } } }
I20260928 15:57:41.330497 30058 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: LEARNER
I20260928 15:57:41.331110 29948 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: attempting to promote NON_VOTER cebe265c0fea44a194e527f18e02d7d5 to VOTER
I20260928 15:57:41.331794 29948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:41.334131 30030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 LEARNER]: Refusing update from remote peer 6991a5163c3e404fb77e798a3f36fee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260928 15:57:41.334172 29831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Refusing update from remote peer 6991a5163c3e404fb77e798a3f36fee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260928 15:57:41.334980 29946 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260928 15:57:41.333235 30078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:41.335354 29947 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260928 15:57:41.335870 30077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:41.337430 29946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
W20260928 15:57:41.337978 30080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:41.338073 29831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:41.339120 30030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:41.339661 29993 server_base.cc:1034] running on GCE node
I20260928 15:57:41.340402 29993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:41.341202 29946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:41.341454 29946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:41.341894 29947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:41.342123 29947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:41.343662 30058 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:41.343753 29947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:41.344005 30058 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:41.344098 29947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:41.349887 30084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6991a5163c3e404fb77e798a3f36fee3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:41.350298 30084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:41.350711 29993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:41.351976 29993 hybrid_clock.cc:648] HybridClock initialized: now 1790611061351901 us; error 69 us; skew 500 ppm
I20260928 15:57:41.352408 29993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:41 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:41.356352 29993 init.cc:377] Logged in from keytab as oryx/127.22.125.193@KRBTEST.COM (short username oryx)
I20260928 15:57:41.361155 29993 webserver.cc:460] Webserver started at http://127.22.125.193:45313/ using document root <none> and password file <none>
I20260928 15:57:41.361887 29993 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:41.362037 29993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:41.367527 29993 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 15:57:41.370352 30090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:41.371650 29993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 15:57:41.371845 29993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "f4c37d8527da493287e4d0702b9600fc"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "347a564bfe00f50e6cb47eabfdf4cdb5"
server_key_iv: "5cb6be2544ccf1e7501526418b458cf3"
server_key_version: "encryptionkey@0"
I20260928 15:57:41.372864 29993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:41.386780 29993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:41.391053 29993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:41.391350 29993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:41.392320 29993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:41.393595 29993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:41.393708 29993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:41.393839 29993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:41.393934 29993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:41.426362 29993 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:42437
I20260928 15:57:41.426450 30202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:42437 every 8 connection(s)
I20260928 15:57:41.427780 29993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:41.430266 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 29993
I20260928 15:57:41.430621 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29402
Sep 28 15:57:41 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.194@KRBTEST.COM
Sep 28 15:57:41 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
Sep 28 15:57:41 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.252@KRBTEST.COM
Sep 28 15:57:41 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.253@KRBTEST.COM
I20260928 15:57:41.479496 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.432204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35154 (local address 127.22.125.194:34925)
0928 15:57:41.452573 (+ 20369us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:41.452587 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:41.453386 (+ 799us) server_negotiation.cc:407] Connection header received
0928 15:57:41.453467 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:41.453475 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:41.453736 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:41.453892 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:41.456814 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.459521 (+ 2707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.461417 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.462454 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.462677 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.469676 (+ 6999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:41.469690 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:41.469706 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:41.469763 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:41.475204 (+ 5441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.476200 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.476213 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.476217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.476440 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.476915 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.476921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.476925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.477381 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:41.477830 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:41.478856 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0928 15:57:41.479128 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20177,"thread_start_us":20022,"threads_started":1}
I20260928 15:57:41.481546 29529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:41.483300 30075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.450024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:48931 (local address 127.22.125.252:45153)
0928 15:57:41.450248 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:41.450257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:41.454851 (+ 4594us) server_negotiation.cc:407] Connection header received
0928 15:57:41.458198 (+ 3347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:41.458203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:41.458294 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:41.458405 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:41.464619 (+ 6214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.465666 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.468073 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.468748 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.468852 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.474948 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:41.474954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:41.474959 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:41.474999 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:41.478639 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.480865 (+ 2226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.480871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.480874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.481044 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.481504 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.481512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.481517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.481858 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:41.481994 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:41.482603 (+ 609us) server_negotiation.cc:299] Negotiation successful
0928 15:57:41.482863 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 15:57:41.483506 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.430737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:49165 (local address 127.22.125.254:37163)
0928 15:57:41.444344 (+ 13607us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:41.444352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:41.447162 (+ 2810us) server_negotiation.cc:407] Connection header received
0928 15:57:41.459050 (+ 11888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:41.459059 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:41.459169 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:41.459279 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:41.463572 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.464895 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.467934 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.468633 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.468736 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.474703 (+ 5967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:41.474710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:41.474716 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:41.474764 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:41.478765 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.480100 (+ 1335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.480105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.480109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.480261 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.480839 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.480845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.480850 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.481239 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:41.481388 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:41.482557 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0928 15:57:41.482886 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13425,"thread_start_us":13617,"threads_started":1}
I20260928 15:57:41.485951 30211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.449900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.22.125.193:48931)
0928 15:57:41.454745 (+ 4845us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:41.454763 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:41.457907 (+ 3144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:41.463598 (+ 5691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:41.463605 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:41.464186 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:41.464453 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.464463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.465845 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.465850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.467874 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.467890 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.469306 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.469316 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.469612 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.470429 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:41.470462 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:41.474749 (+ 4287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:41.478864 (+ 4115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:41.478875 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:41.478896 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:41.480315 (+ 1419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:41.481190 (+ 875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:41.481195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:41.481199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:41.481369 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:41.482128 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:41.482132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:41.482288 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 15:57:41.483941 (+ 1653us) client_negotiation.cc:240] Negotiation successful
0928 15:57:41.484432 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4704,"mutex_wait_us":322,"thread_start_us":6087,"threads_started":1}
I20260928 15:57:41.486296 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.431619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:49165)
0928 15:57:41.446933 (+ 15314us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:41.447004 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:41.458903 (+ 11899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:41.459563 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:41.459579 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:41.460445 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:41.463392 (+ 2947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.463421 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.465116 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.465122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.467699 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.467714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.469538 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.469547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.469815 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.470595 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:41.470617 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:41.474433 (+ 3816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:41.479256 (+ 4823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:41.479261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:41.479266 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:41.479809 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:41.480431 (+ 622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:41.480438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:41.480442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:41.480683 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:41.481501 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:41.481510 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:41.482022 (+ 512us) client_negotiation.cc:769] Sending connection context
0928 15:57:41.483765 (+ 1743us) client_negotiation.cc:240] Negotiation successful
0928 15:57:41.485006 (+ 1241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14809,"mutex_wait_us":710,"thread_start_us":21266,"threads_started":1}
I20260928 15:57:41.486179 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.454160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.22.125.193:46083)
0928 15:57:41.463328 (+ 9168us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:41.463350 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:41.463450 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:41.468395 (+ 4945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:41.468400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:41.468899 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:41.469404 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.469417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.470823 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.470831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.472441 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:41.472455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.473627 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.473633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:41.473835 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.474979 (+ 1144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:41.474998 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:41.478455 (+ 3457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:41.482755 (+ 4300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:41.482762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:41.482768 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:41.483287 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:41.483891 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:41.483896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:41.483899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:41.484152 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:41.484833 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:41.484849 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:41.484998 (+ 149us) client_negotiation.cc:769] Sending connection context
0928 15:57:41.485147 (+ 149us) client_negotiation.cc:240] Negotiation successful
0928 15:57:41.485412 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9058,"thread_start_us":8912,"threads_started":1}
I20260928 15:57:41.487893 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:41.454339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:46083 (local address 127.22.125.253:38453)
0928 15:57:41.467870 (+ 13531us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:41.467879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:41.467900 (+ 21us) server_negotiation.cc:407] Connection header received
0928 15:57:41.467959 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:41.467964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:41.468215 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:41.469255 (+ 1040us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:41.469579 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.470638 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.472638 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:41.473231 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:41.473360 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:41.478734 (+ 5374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:41.478743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:41.478749 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:41.478790 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:41.482614 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.483490 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.483495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.483498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.483738 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:41.484281 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:41.484285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:41.484288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:41.484624 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:41.484773 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:41.485980 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0928 15:57:41.487526 (+ 1546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13360,"thread_start_us":13200,"threads_started":1}
I20260928 15:57:41.488168 30204 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37163
I20260928 15:57:41.488441 30204 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:41.488989 30205 heartbeater.cc:347] Connected to a master server at 127.22.125.253:38453
I20260928 15:57:41.489115 30205 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:41.489472 30204 heartbeater.cc:511] Master 127.22.125.254:37163 requested a full tablet report, sending...
I20260928 15:57:41.489586 30205 heartbeater.cc:511] Master 127.22.125.253:38453 requested a full tablet report, sending...
I20260928 15:57:41.489842 30203 heartbeater.cc:347] Connected to a master server at 127.22.125.252:45153
I20260928 15:57:41.489952 30203 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:41.490419 30203 heartbeater.cc:511] Master 127.22.125.252:45153 requested a full tablet report, sending...
I20260928 15:57:41.491693 29205 ts_manager.cc:194] Re-registered known tserver with Master: f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437)
I20260928 15:57:41.492093 29821 ts_manager.cc:194] Registered new tserver with Master: f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437)
I20260928 15:57:41.492547 30019 ts_manager.cc:194] Registered new tserver with Master: f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437)
I20260928 15:57:41.492851 29205 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:49165
I20260928 15:57:41.574718 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29402
I20260928 15:57:41.592118 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:34925
--local_ip_for_outbound_sockets=127.22.125.194
--tserver_master_addrs=127.22.125.254:37163
--webserver_port=33727
--webserver_interface=127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab
--principal=oryx/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:41.920854 30215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:41.921137 30215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:41.921319 30215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:41.929167 30215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:41.929272 30215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:41.929441 30215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:41.939656 30215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab
--principal=oryx/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:34925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=33727
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:41.941030 30215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:41.942358 30215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:41.955726 30223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:41.955802 30221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:41.955760 30220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:41.957242 30215 server_base.cc:1034] running on GCE node
I20260928 15:57:41.957830 30215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:41.958765 30215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:41.959983 30215 hybrid_clock.cc:648] HybridClock initialized: now 1790611061959938 us; error 37 us; skew 500 ppm
I20260928 15:57:41.960317 30215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:41 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:41.963584 30215 init.cc:377] Logged in from keytab as oryx/127.22.125.194@KRBTEST.COM (short username oryx)
I20260928 15:57:41.964998 30215 webserver.cc:460] Webserver started at http://127.22.125.194:33727/ using document root <none> and password file <none>
I20260928 15:57:41.965660 30215 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:41.965803 30215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:41.971129 30215 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 15:57:41.973794 30230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:41.975065 30215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:57:41.975229 30215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "718611922ca04e478eef8495c05fb4f7"
format_stamp: "Formatted at 2026-09-28 15:57:38 on dist-test-slave-9cqd"
server_key: "fc3e089c72443ba437893cac9d7d3999"
server_key_iv: "0e4e6be711dc15071ef6003717fd38df"
server_key_version: "encryptionkey@0"
I20260928 15:57:41.976253 30215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:41.992780 30215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:41.996546 30215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:41.996829 30215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:41.997726 30215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:41.999142 30215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:41.999228 30215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:41.999372 30215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:41.999454 30215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:42.032253 30215 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:34925
I20260928 15:57:42.032325 30342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:34925 every 8 connection(s)
I20260928 15:57:42.033793 30215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:42.041950 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 30215
I20260928 15:57:42.042243 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29539
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.252@KRBTEST.COM
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.195@KRBTEST.COM
I20260928 15:57:42.093370 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.037267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:60043)
0928 15:57:42.066579 (+ 29312us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.066646 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.070079 (+ 3433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.072180 (+ 2101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.072192 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.073125 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:42.073893 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.073917 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.075390 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.075396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.077483 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.077498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.078761 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.078768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.079054 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.079890 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:42.079920 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:42.083106 (+ 3186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:42.088119 (+ 5013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.088132 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.088152 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.088782 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.089252 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.089257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.089260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.089542 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.090305 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:42.090313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:42.090821 (+ 508us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.091169 (+ 348us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.091751 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28896,"thread_start_us":33272,"threads_started":1}
I20260928 15:57:42.094753 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.036619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:60043 (local address 127.22.125.254:37163)
0928 15:57:42.071603 (+ 34984us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.071608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.071626 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:57:42.071684 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.071688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.071774 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.071887 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.074115 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.075200 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.077683 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.078357 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.078485 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.083336 (+ 4851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.083342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.083346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.083387 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.087903 (+ 4516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.088951 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.088957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.088960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.089118 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.089689 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.089697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.089702 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.090087 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.090232 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.094143 (+ 3911us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.094386 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34872,"thread_start_us":34768,"threads_started":1}
I20260928 15:57:42.097455 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.037020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:44789 (local address 127.22.125.252:45153)
0928 15:57:42.060909 (+ 23889us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.060918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.071099 (+ 10181us) server_negotiation.cc:407] Connection header received
0928 15:57:42.071458 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.071466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.071591 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.071723 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.076139 (+ 4416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.077290 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.082586 (+ 5296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.083234 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.083322 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.089437 (+ 6115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.089447 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.089454 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.089510 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.093948 (+ 4438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.095004 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.095009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.095012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.095176 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.095730 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.095735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.095738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.096154 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.096292 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.096814 (+ 522us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.097053 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23703,"thread_start_us":23886,"threads_started":1}
I20260928 15:57:42.098381 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.037268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.22.125.194:44789)
0928 15:57:42.070992 (+ 33724us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.071016 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.071160 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.072180 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.072192 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.074803 (+ 2611us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:42.075101 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.075112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.078088 (+ 2976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.078095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.079580 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.079593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.083694 (+ 4101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.083700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.083870 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.084748 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:42.084770 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:42.089152 (+ 4382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:42.094148 (+ 4996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.094155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.094162 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.094869 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.095383 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.095391 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.095397 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.095582 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.096362 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:42.096370 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:42.096526 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.097653 (+ 1127us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.097954 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33531,"mutex_wait_us":297,"thread_start_us":28463,"threads_started":1}
I20260928 15:57:42.099905 30343 heartbeater.cc:347] Connected to a master server at 127.22.125.252:45153
I20260928 15:57:42.100244 30343 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:42.101024 30343 heartbeater.cc:511] Master 127.22.125.252:45153 requested a full tablet report, sending...
I20260928 15:57:42.102144 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.043420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.22.125.195:44653)
0928 15:57:42.064977 (+ 21557us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.064989 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.065692 (+ 703us) server_negotiation.cc:407] Connection header received
0928 15:57:42.065898 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.065912 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.066274 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.066447 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.069300 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.072633 (+ 3333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.075277 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.077719 (+ 2442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.077829 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.091532 (+ 13703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.091548 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.091563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.091633 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.095595 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.096984 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.096992 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.096996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.097156 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.097662 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.097667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.097670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.098068 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.098362 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.099561 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.099822 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21319,"thread_start_us":21154,"threads_started":1}
I20260928 15:57:42.102696 30019 ts_manager.cc:194] Registered new tserver with Master: 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925)
I20260928 15:57:42.103495 29665 generic_service.cc:196] Checking for leaks (request via RPC)
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611061, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.253@KRBTEST.COM
I20260928 15:57:42.108932 30344 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37163
I20260928 15:57:42.109086 30344 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:42.109473 30344 heartbeater.cc:511] Master 127.22.125.254:37163 requested a full tablet report, sending...
I20260928 15:57:42.110513 29205 ts_manager.cc:194] Re-registered known tserver with Master: 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925)
I20260928 15:57:42.111694 29205 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:60043
I20260928 15:57:42.114423 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.071230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.22.125.194:41947)
0928 15:57:42.093818 (+ 22588us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.093853 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.095049 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.096112 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.096120 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.096767 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:42.097004 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.097018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.099498 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.099506 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.101737 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.101757 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.103297 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.103306 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.103571 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.104288 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:42.104319 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:42.107517 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:42.111707 (+ 4190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.111715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.111721 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.112257 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.112695 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.112701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.112705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.112859 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.113409 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:42.113414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:42.113553 (+ 139us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.113716 (+ 163us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.114025 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22463,"thread_start_us":36722,"threads_started":1}
I20260928 15:57:42.114415 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.071235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:41947 (local address 127.22.125.253:38453)
0928 15:57:42.088435 (+ 17200us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.088441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.095744 (+ 7303us) server_negotiation.cc:407] Connection header received
0928 15:57:42.095825 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.095830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.095922 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.096054 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.097222 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.099136 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.101953 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.102869 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.103016 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.107816 (+ 4800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.107826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.107832 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.107884 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.111539 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.112413 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.112418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.112421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.112569 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.112974 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.112979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.112983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.113275 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.113435 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.113833 (+ 398us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.114034 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17060,"thread_start_us":16959,"threads_started":1}
I20260928 15:57:42.115258 30345 heartbeater.cc:347] Connected to a master server at 127.22.125.253:38453
I20260928 15:57:42.115425 30345 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:42.115886 30345 heartbeater.cc:511] Master 127.22.125.253:38453 requested a full tablet report, sending...
I20260928 15:57:42.116904 29821 ts_manager.cc:194] Registered new tserver with Master: 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925)
I20260928 15:57:42.189020 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29539
I20260928 15:57:42.204828 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:44653
--local_ip_for_outbound_sockets=127.22.125.195
--tserver_master_addrs=127.22.125.254:37163
--webserver_port=40457
--webserver_interface=127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab
--principal=oryx/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 15:57:42.495831 30204 heartbeater.cc:503] Master 127.22.125.254:37163 was elected leader, sending a full tablet report...
W20260928 15:57:42.544807 30356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:42.545110 30356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:42.545287 30356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:42.552754 30356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:42.552886 30356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:42.553058 30356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:42.562366 30356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab
--principal=oryx/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:44653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=40457
--tserver_master_addrs=127.22.125.254:37163,127.22.125.253:38453,127.22.125.252:45153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:42.563683 30356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:42.565100 30356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:42.578532 30361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:42.578558 30364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:42.578486 30362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:42.579303 30356 server_base.cc:1034] running on GCE node
I20260928 15:57:42.579945 30356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:42.581017 30356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:42.582235 30356 hybrid_clock.cc:648] HybridClock initialized: now 1790611062582181 us; error 44 us; skew 500 ppm
I20260928 15:57:42.582548 30356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611062, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:42.585687 30356 init.cc:377] Logged in from keytab as oryx/127.22.125.195@KRBTEST.COM (short username oryx)
I20260928 15:57:42.587064 30356 webserver.cc:460] Webserver started at http://127.22.125.195:40457/ using document root <none> and password file <none>
I20260928 15:57:42.587740 30356 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:42.587901 30356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:42.593116 30356 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:42.596304 30371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:42.597594 30356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 15:57:42.597769 30356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "d3db5ab25c94440a80c3ab45878cd12a"
format_stamp: "Formatted at 2026-09-28 15:57:39 on dist-test-slave-9cqd"
server_key: "4b69345ff1176f017e3d2223bf5abdc6"
server_key_iv: "aca90b6c46218393742c79d037d7ef71"
server_key_version: "encryptionkey@0"
I20260928 15:57:42.598773 30356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:42.617578 30356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:42.621976 30356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:42.622270 30356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:42.623185 30356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:42.624519 30356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:42.624641 30356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:42.624771 30356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:42.624856 30356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:42.660933 30356 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:44653
I20260928 15:57:42.660993 30483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:44653 every 8 connection(s)
I20260928 15:57:42.662513 30356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:42.664767 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 30356
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611062, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.252@KRBTEST.COM
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611062, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.253@KRBTEST.COM
Sep 28 15:57:42 dist-test-slave-9cqd krb5kdc[29160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611062, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:42.710566 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.671473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:48293 (local address 127.22.125.252:45153)
0928 15:57:42.672104 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.672113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.678174 (+ 6061us) server_negotiation.cc:407] Connection header received
0928 15:57:42.679479 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.679483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.679550 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.679668 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.682559 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.683389 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.686380 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.686765 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.686900 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.692164 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.692172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.692179 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.692229 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.696860 (+ 4631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.702391 (+ 5531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.702397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.702401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.702559 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.708502 (+ 5943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.708507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.708510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.708887 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.709032 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.709938 (+ 906us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.710174 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":287,"threads_started":1}
I20260928 15:57:42.711889 30494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.670981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.22.125.195:48293)
0928 15:57:42.678083 (+ 7102us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.678100 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.679089 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.679980 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.679992 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.681041 (+ 1049us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:42.682370 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.682404 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.683548 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.683557 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.686244 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.686258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.686980 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.686987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.687256 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.688031 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:42.688093 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:42.691887 (+ 3794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:42.698654 (+ 6767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.698663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.698683 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.700275 (+ 1592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.702913 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.702917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.702920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.708387 (+ 5467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.709009 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:42.709017 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:42.709472 (+ 455us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.709820 (+ 348us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.710330 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6966,"mutex_wait_us":172,"spinlock_wait_cycles":1280,"thread_start_us":12769,"threads_started":1}
I20260928 15:57:42.712944 30488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.665557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:53463 (local address 127.22.125.253:38453)
0928 15:57:42.676485 (+ 10928us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.676490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.676855 (+ 365us) server_negotiation.cc:407] Connection header received
0928 15:57:42.679377 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.679386 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.679499 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.679636 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.682623 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.684004 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.686683 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.687601 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.687723 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.698121 (+ 10398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.698131 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.698152 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.698217 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.702244 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.709326 (+ 7082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.709331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.709334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.709497 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.711170 (+ 1673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.711175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.711178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.711514 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.711677 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.712320 (+ 643us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.712563 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10647,"thread_start_us":10751,"threads_started":1}
I20260928 15:57:42.713949 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.666152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.22.125.195:53463)
0928 15:57:42.676641 (+ 10489us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.676751 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.679052 (+ 2301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.679980 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.679992 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.681088 (+ 1096us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:42.682370 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.682403 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.684219 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.684234 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.686479 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.686499 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.688111 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.688120 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.688420 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.689153 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:42.689183 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:42.697852 (+ 8669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:42.702761 (+ 4909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.702765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.702769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.709192 (+ 6423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.709656 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.709660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.709664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.711024 (+ 1360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.711759 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:42.711765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:42.711978 (+ 213us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.713222 (+ 1244us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.713525 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10185,"mutex_wait_us":144,"spinlock_wait_cycles":15616,"thread_start_us":9967,"threads_started":1}
I20260928 15:57:42.714123 30484 heartbeater.cc:347] Connected to a master server at 127.22.125.252:45153
I20260928 15:57:42.714557 30484 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:42.715567 30484 heartbeater.cc:511] Master 127.22.125.252:45153 requested a full tablet report, sending...
I20260928 15:57:42.717576 30019 ts_manager.cc:194] Registered new tserver with Master: d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:42.717698 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.677435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:53689)
0928 15:57:42.684074 (+ 6639us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.684095 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.684274 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.684839 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.684848 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.685454 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:42.685840 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.685852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.688094 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.688099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.689840 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.689857 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.690596 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.690602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.690799 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.692322 (+ 1523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:42.692347 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:42.702766 (+ 10419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:42.714479 (+ 11713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.714483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.714488 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.714977 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.715891 (+ 914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:42.715897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:42.715902 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:42.716094 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:42.716751 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:42.716756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:42.716911 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.717039 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.717313 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6494,"thread_start_us":273,"threads_started":1}
I20260928 15:57:42.718680 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.677555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:53689 (local address 127.22.125.254:37163)
0928 15:57:42.682166 (+ 4611us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.682173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.684409 (+ 2236us) server_negotiation.cc:407] Connection header received
0928 15:57:42.684487 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.684491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.684569 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.684670 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.686018 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.687889 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.690034 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.690414 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.690560 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.702941 (+ 12381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.702949 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.702956 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.703010 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.707960 (+ 4950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.715569 (+ 7609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.715575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.715579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.715747 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.716260 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.716265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.716268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.716602 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.717933 (+ 1331us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.718075 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.718322 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4475,"thread_start_us":4371,"threads_started":1}
I20260928 15:57:42.719599 30485 heartbeater.cc:347] Connected to a master server at 127.22.125.254:37163
I20260928 15:57:42.719740 30485 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:42.720257 30485 heartbeater.cc:511] Master 127.22.125.254:37163 requested a full tablet report, sending...
I20260928 15:57:42.722332 29205 ts_manager.cc:194] Re-registered known tserver with Master: d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:42.723502 29205 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:53689
I20260928 15:57:42.727167 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.692997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.22.125.252:45153)
0928 15:57:42.705100 (+ 12103us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.705108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.705822 (+ 714us) server_negotiation.cc:407] Connection header received
0928 15:57:42.706096 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.706105 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.706240 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.706338 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.709418 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.711065 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.713286 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.714122 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.714228 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.718701 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.718710 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.718718 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.718767 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.723129 (+ 4362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.724230 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.724238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.724244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.724435 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.725070 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.725078 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.725084 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.725527 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.725817 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.726542 (+ 725us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.726793 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11910,"thread_start_us":12116,"threads_started":1}
I20260928 15:57:42.729130 30486 heartbeater.cc:347] Connected to a master server at 127.22.125.253:38453
I20260928 15:57:42.729265 30486 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:42.729627 30486 heartbeater.cc:511] Master 127.22.125.253:38453 requested a full tablet report, sending...
I20260928 15:57:42.730880 29821 ts_manager.cc:194] Registered new tserver with Master: d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:42.737596 30488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.705872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.22.125.253:38453)
0928 15:57:42.713175 (+ 7303us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.713180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.719929 (+ 6749us) server_negotiation.cc:407] Connection header received
0928 15:57:42.720224 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.720231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.720342 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.720415 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.723032 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.724017 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.725908 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.726478 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.726569 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.731391 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.731404 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.731409 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.731437 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.734797 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.735623 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.735627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.735629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.735753 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.736154 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.736160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.736163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.736429 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.736694 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.737082 (+ 388us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.737296 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7191,"thread_start_us":22529,"threads_started":1}
I20260928 15:57:42.740600 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.692825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35304 (local address 127.22.125.254:37163)
0928 15:57:42.708582 (+ 15757us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.708590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.708608 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:57:42.708664 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.708668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.708734 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.708835 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:42.729013 (+ 20178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.729893 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.731460 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.731928 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.732086 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.735047 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:42.735054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:42.735079 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:42.735117 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:42.738261 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.739019 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.739022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.739025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.739135 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:42.739439 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:42.739442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:42.739444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:42.739697 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:42.739859 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.740175 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.740332 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15177,"thread_start_us":15266,"threads_started":1}
I20260928 15:57:42.744221 29205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:42.751236 29205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:42.781804 30515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.768997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:42437 (local address 127.0.0.1:45056)
0928 15:57:42.770645 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.770665 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.770804 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.771943 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.771952 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.771987 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:42.772445 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.772461 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.774717 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.774725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.777813 (+ 3088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.777834 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.780686 (+ 2852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.780694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.780947 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.781002 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.781108 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.781326 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1516,"thread_start_us":1378,"threads_started":1}
I20260928 15:57:42.783710 30516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.769315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.22.125.193:42437)
0928 15:57:42.770170 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.770200 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.770956 (+ 756us) server_negotiation.cc:407] Connection header received
0928 15:57:42.771113 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.771123 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.771423 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.771638 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:42.772654 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.774519 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.778109 (+ 3590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.780058 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.780261 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.780499 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.782850 (+ 2351us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.783094 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":584,"threads_started":1}
I20260928 15:57:42.787741 30138 tablet_service.cc:1467] Processing CreateTablet for tablet b04776c3b6f44cffaa1d851a7395528b (DEFAULT_TABLE table=test-table [id=66cb7e3d472148b98cb5fd872b7058db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:42.789028 30138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04776c3b6f44cffaa1d851a7395528b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:42.796247 30515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.769812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44653 (local address 127.0.0.1:45906)
0928 15:57:42.782511 (+ 12699us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.782536 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.782662 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.790414 (+ 7752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.790419 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.790444 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:42.790665 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.790679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.792564 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.792570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.793752 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.793765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.795462 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.795468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.795632 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.795674 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.795769 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.795912 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12526,"thread_start_us":12450,"threads_started":1}
I20260928 15:57:42.796326 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.770502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.22.125.195:44653)
0928 15:57:42.789957 (+ 19455us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.789963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.789988 (+ 25us) server_negotiation.cc:407] Connection header received
0928 15:57:42.790062 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.790067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.790214 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.790428 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:42.790800 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.792445 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.793920 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.795303 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.795472 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.795638 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.795825 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.795973 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19291,"thread_start_us":19133,"threads_started":1}
I20260928 15:57:42.797449 30517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.770760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:34925 (local address 127.0.0.1:47082)
0928 15:57:42.782588 (+ 11828us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.782602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.782694 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.789268 (+ 6574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.789273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.789301 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:42.789630 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.789642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.791432 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.791438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.794938 (+ 3500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.794953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.796722 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.796728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.796853 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.796882 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.796992 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.797117 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11680,"thread_start_us":12171,"threads_started":1}
I20260928 15:57:42.797533 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.771082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.22.125.194:34925)
0928 15:57:42.788508 (+ 17426us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.788521 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.788559 (+ 38us) server_negotiation.cc:407] Connection header received
0928 15:57:42.788681 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.788693 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.789030 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.789267 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:42.789867 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.791279 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.795125 (+ 3846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.796515 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.796665 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.796829 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.797043 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.797219 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17252,"thread_start_us":17144,"threads_started":1}
I20260928 15:57:42.801433 30278 tablet_service.cc:1467] Processing CreateTablet for tablet b04776c3b6f44cffaa1d851a7395528b (DEFAULT_TABLE table=test-table [id=66cb7e3d472148b98cb5fd872b7058db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:42.802196 30419 tablet_service.cc:1467] Processing CreateTablet for tablet b04776c3b6f44cffaa1d851a7395528b (DEFAULT_TABLE table=test-table [id=66cb7e3d472148b98cb5fd872b7058db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:42.802949 30278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04776c3b6f44cffaa1d851a7395528b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:42.803488 30521 tablet_bootstrap.cc:492] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Bootstrap starting.
I20260928 15:57:42.803596 30419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04776c3b6f44cffaa1d851a7395528b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:42.807160 30521 tablet_bootstrap.cc:654] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:42.808535 30521 log.cc:824] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:42.811664 30521 tablet_bootstrap.cc:492] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: No bootstrap required, opened a new log
I20260928 15:57:42.811954 30521 ts_tablet_manager.cc:1397] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 15:57:42.815672 30523 tablet_bootstrap.cc:492] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Bootstrap starting.
I20260928 15:57:42.818794 30523 tablet_bootstrap.cc:654] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:42.819887 30524 tablet_bootstrap.cc:492] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Bootstrap starting.
I20260928 15:57:42.820257 30523 log.cc:824] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:42.823261 30521 raft_consensus.cc:348] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.823488 30523 tablet_bootstrap.cc:492] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: No bootstrap required, opened a new log
I20260928 15:57:42.823535 30524 tablet_bootstrap.cc:654] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:42.823637 30521 raft_consensus.cc:374] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:42.823747 30523 ts_tablet_manager.cc:1397] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260928 15:57:42.823745 30521 raft_consensus.cc:729] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c37d8527da493287e4d0702b9600fc, State: Initialized, Role: FOLLOWER
I20260928 15:57:42.824361 30521 consensus_queue.cc:260] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.824733 30524 log.cc:824] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:42.826081 30521 ts_tablet_manager.cc:1428] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260928 15:57:42.838807 30524 tablet_bootstrap.cc:492] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: No bootstrap required, opened a new log
I20260928 15:57:42.839118 30524 ts_tablet_manager.cc:1397] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20260928 15:57:42.839378 30523 raft_consensus.cc:348] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.839864 30523 raft_consensus.cc:374] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:42.839969 30523 raft_consensus.cc:729] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718611922ca04e478eef8495c05fb4f7, State: Initialized, Role: FOLLOWER
I20260928 15:57:42.840634 30523 consensus_queue.cc:260] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.841684 30344 heartbeater.cc:503] Master 127.22.125.254:37163 was elected leader, sending a full tablet report...
I20260928 15:57:42.842389 30523 ts_tablet_manager.cc:1428] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
I20260928 15:57:42.848587 30524 raft_consensus.cc:348] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.849054 30524 raft_consensus.cc:374] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:42.849169 30524 raft_consensus.cc:729] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3db5ab25c94440a80c3ab45878cd12a, State: Initialized, Role: FOLLOWER
I20260928 15:57:42.849706 30524 consensus_queue.cc:260] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.850693 30485 heartbeater.cc:503] Master 127.22.125.254:37163 was elected leader, sending a full tablet report...
I20260928 15:57:42.851354 30524 ts_tablet_manager.cc:1428] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260928 15:57:42.921339 30487 tablet.cc:2369] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:57:42.938748 30526 raft_consensus.cc:482] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:42.939075 30526 raft_consensus.cc:504] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.940845 30526 leader_election.cc:288] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925), d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:42.947351 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.941131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:36533 (local address 127.22.125.194:34925)
0928 15:57:42.941358 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.941367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.942078 (+ 711us) server_negotiation.cc:407] Connection header received
0928 15:57:42.942319 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.942323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.942387 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.942481 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:42.943041 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.943732 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.945631 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.946407 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.946615 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.946692 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.946970 (+ 278us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.947094 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 15:57:42.947383 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.941381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:48699 (local address 127.22.125.195:44653)
0928 15:57:42.941632 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:42.941639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:42.942348 (+ 709us) server_negotiation.cc:407] Connection header received
0928 15:57:42.942560 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:42.942564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:42.942621 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:42.942716 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:42.943235 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.943949 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.945707 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.946467 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.946647 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.946732 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:42.947015 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 15:57:42.947142 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 15:57:42.947489 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.941076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:34925 (local address 127.22.125.193:36533)
0928 15:57:42.941956 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.941971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.942162 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.942496 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.942501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.942526 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:42.942883 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.942892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.943877 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.943884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.945453 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.945466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.946638 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.946644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.946807 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.946830 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.946945 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.947070 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":2,"thread_start_us":486,"threads_started":1}
I20260928 15:57:42.948103 30531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:42.941146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44653 (local address 127.22.125.193:48699)
0928 15:57:42.942242 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:42.942256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:42.942413 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:42.942737 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:42.942741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:42.942767 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:42.943088 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.943096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.944115 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.944120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.945555 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:42.945567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:42.946721 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:42.946727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:42.946863 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:42.946885 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 15:57:42.946989 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 15:57:42.947114 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":1,"thread_start_us":789,"threads_started":1}
I20260928 15:57:42.948493 30298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04776c3b6f44cffaa1d851a7395528b" candidate_uuid: "f4c37d8527da493287e4d0702b9600fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "718611922ca04e478eef8495c05fb4f7" is_pre_election: true
I20260928 15:57:42.948983 30298 raft_consensus.cc:2454] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c37d8527da493287e4d0702b9600fc in term 0.
I20260928 15:57:42.949080 30439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04776c3b6f44cffaa1d851a7395528b" candidate_uuid: "f4c37d8527da493287e4d0702b9600fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3db5ab25c94440a80c3ab45878cd12a" is_pre_election: true
I20260928 15:57:42.949579 30092 leader_election.cc:302] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 718611922ca04e478eef8495c05fb4f7, f4c37d8527da493287e4d0702b9600fc; no voters:
I20260928 15:57:42.949604 30439 raft_consensus.cc:2454] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c37d8527da493287e4d0702b9600fc in term 0.
I20260928 15:57:42.949954 30526 raft_consensus.cc:2781] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:42.950141 30526 raft_consensus.cc:482] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:42.950256 30526 raft_consensus.cc:3037] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:42.952617 30526 raft_consensus.cc:504] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
I20260928 15:57:42.953171 30526 leader_election.cc:288] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [CANDIDATE]: Term 1 election: Requested vote from peers 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925), d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:42.953542 30298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04776c3b6f44cffaa1d851a7395528b" candidate_uuid: "f4c37d8527da493287e4d0702b9600fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "718611922ca04e478eef8495c05fb4f7"
I20260928 15:57:42.953593 30439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04776c3b6f44cffaa1d851a7395528b" candidate_uuid: "f4c37d8527da493287e4d0702b9600fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3db5ab25c94440a80c3ab45878cd12a"
I20260928 15:57:42.953729 30298 raft_consensus.cc:3037] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:42.953791 30439 raft_consensus.cc:3037] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:42.955907 30298 raft_consensus.cc:2454] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c37d8527da493287e4d0702b9600fc in term 1.
I20260928 15:57:42.955945 30439 raft_consensus.cc:2454] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c37d8527da493287e4d0702b9600fc in term 1.
I20260928 15:57:42.956403 30091 leader_election.cc:302] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3db5ab25c94440a80c3ab45878cd12a, f4c37d8527da493287e4d0702b9600fc; no voters:
I20260928 15:57:42.956703 30526 raft_consensus.cc:2781] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:42.957106 30526 raft_consensus.cc:686] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 1 LEADER]: Becoming Leader. State: Replica: f4c37d8527da493287e4d0702b9600fc, State: Running, Role: LEADER
I20260928 15:57:42.957686 30526 consensus_queue.cc:237] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } }
W20260928 15:57:42.963364 30206 tablet.cc:2369] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:57:42.963135 29205 catalog_manager.cc:5495] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc reported cstate change: term changed from 0 to 1, leader changed from <none> to f4c37d8527da493287e4d0702b9600fc (127.22.125.193). New cstate: current_term: 1 leader_uuid: "f4c37d8527da493287e4d0702b9600fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:43.017885 30516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:43.011882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45068 (local address 127.22.125.193:42437)
0928 15:57:43.012189 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:43.012195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:43.012219 (+ 24us) server_negotiation.cc:407] Connection header received
0928 15:57:43.012456 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:43.012461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:43.012547 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:43.012738 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:43.013178 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.013975 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.015473 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.015930 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.016099 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:43.016603 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:43.017243 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:43.017362 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:43.017543 (+ 181us) server_negotiation.cc:299] Negotiation successful
0928 15:57:43.017606 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 15:57:43.025894 30439 raft_consensus.cc:1261] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Refusing update from remote peer f4c37d8527da493287e4d0702b9600fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:43.025894 30298 raft_consensus.cc:1261] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 1 FOLLOWER]: Refusing update from remote peer f4c37d8527da493287e4d0702b9600fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:43.027060 30526 consensus_queue.cc:1035] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:43.027402 30532 consensus_queue.cc:1035] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:43.044431 30536 mvcc.cc:205] Tried to move back new op lower bound from 7334342914143739904 to 7334342913878335488. Current Snapshot: MvccSnapshot[applied={T|T < 7334342914143739904 or (T in {7334342914143739904})}]
I20260928 15:57:43.068329 29204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:43.068647 29204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:43.075505 29204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6991a5163c3e404fb77e798a3f36fee3: Sending DeleteTablet for 3 replicas of tablet b04776c3b6f44cffaa1d851a7395528b
I20260928 15:57:43.076983 30278 tablet_service.cc:1514] Processing DeleteTablet for tablet b04776c3b6f44cffaa1d851a7395528b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:43 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:47082
I20260928 15:57:43.077090 30138 tablet_service.cc:1514] Processing DeleteTablet for tablet b04776c3b6f44cffaa1d851a7395528b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:43 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:45056
I20260928 15:57:43.077885 30419 tablet_service.cc:1514] Processing DeleteTablet for tablet b04776c3b6f44cffaa1d851a7395528b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:43 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:45906
I20260928 15:57:43.078030 30545 tablet_replica.cc:330] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: stopping tablet replica
I20260928 15:57:43.078492 30545 raft_consensus.cc:2229] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:43.078979 30545 raft_consensus.cc:2258] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:43.082751 30545 ts_tablet_manager.cc:1905] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:43.091312 30545 ts_tablet_manager.cc:1918] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:43.091493 30545 log.cc:1196] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/b04776c3b6f44cffaa1d851a7395528b
I20260928 15:57:43.092131 30545 ts_tablet_manager.cc:1939] T b04776c3b6f44cffaa1d851a7395528b P 718611922ca04e478eef8495c05fb4f7: Deleting consensus metadata
I20260928 15:57:43.094190 29190 catalog_manager.cc:4841] TS 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925): tablet b04776c3b6f44cffaa1d851a7395528b (table test-table [id=66cb7e3d472148b98cb5fd872b7058db]) successfully deleted
I20260928 15:57:43.100600 30547 tablet_replica.cc:330] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: stopping tablet replica
I20260928 15:57:43.101172 30547 raft_consensus.cc:2229] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:43.101787 30547 raft_consensus.cc:2258] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:43.102663 30546 tablet_replica.cc:330] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: stopping tablet replica
I20260928 15:57:43.103091 30546 raft_consensus.cc:2229] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:43.103703 30546 raft_consensus.cc:2258] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:43.105542 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29173
I20260928 15:57:43.106748 29236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:43.106915 30547 ts_tablet_manager.cc:1905] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:43.106936 30546 ts_tablet_manager.cc:1905] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:43.114348 30546 ts_tablet_manager.cc:1918] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:43.115034 30546 log.cc:1196] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/b04776c3b6f44cffaa1d851a7395528b
I20260928 15:57:43.115759 30546 ts_tablet_manager.cc:1939] T b04776c3b6f44cffaa1d851a7395528b P f4c37d8527da493287e4d0702b9600fc: Deleting consensus metadata
I20260928 15:57:43.118724 30547 ts_tablet_manager.cc:1918] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:43.119029 30547 log.cc:1196] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/b04776c3b6f44cffaa1d851a7395528b
I20260928 15:57:43.119593 30547 ts_tablet_manager.cc:1939] T b04776c3b6f44cffaa1d851a7395528b P d3db5ab25c94440a80c3ab45878cd12a: Deleting consensus metadata
I20260928 15:57:43.133692 30488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:43.081258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.22.125.253:38453)
0928 15:57:43.081452 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:43.081460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:43.105640 (+ 24180us) server_negotiation.cc:407] Connection header received
0928 15:57:43.105723 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:43.105728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:43.105815 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:43.105921 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:43.110272 (+ 4351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.111492 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.117621 (+ 6129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.118269 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.118420 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:43.127965 (+ 9545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:43.127970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:43.127973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:43.128009 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:43.130901 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.131576 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.131581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.131583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.131712 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.132128 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.132133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.132138 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.132442 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:43.132702 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:43.133058 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 15:57:43.133293 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 15:57:43.146572 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:43.104772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.22.125.252:45153)
0928 15:57:43.105017 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:43.105024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:43.133803 (+ 28779us) server_negotiation.cc:407] Connection header received
0928 15:57:43.134105 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:43.134111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:43.134203 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:43.134282 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:43.136190 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.136890 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.138068 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.138528 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.138668 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:43.141155 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:43.141160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:43.141163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:43.141193 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:43.143841 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.144578 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.144590 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.144593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.144720 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.145116 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.145120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.145122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.145384 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:43.145631 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:43.145989 (+ 358us) server_negotiation.cc:299] Negotiation successful
0928 15:57:43.146230 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 15:57:43.266232 29190 catalog_manager.cc:4841] TS f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437): tablet b04776c3b6f44cffaa1d851a7395528b (table test-table [id=66cb7e3d472148b98cb5fd872b7058db]) successfully deleted
I20260928 15:57:43.266903 29189 catalog_manager.cc:4841] TS d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653): tablet b04776c3b6f44cffaa1d851a7395528b (table test-table [id=66cb7e3d472148b98cb5fd872b7058db]) successfully deleted
I20260928 15:57:43.267547 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29173
I20260928 15:57:43.311581 30488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:43.294466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.22.125.253:38453)
0928 15:57:43.294651 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:43.294657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:43.295436 (+ 779us) server_negotiation.cc:407] Connection header received
0928 15:57:43.295759 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:43.295767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:43.295852 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:43.295973 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:43.298157 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.299189 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.300683 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.301375 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.301505 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:43.305094 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:43.305100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:43.305105 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:43.305150 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:43.308556 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.309351 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.309357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.309360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.309527 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.309959 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.309968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.309974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.310351 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:43.310582 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:43.311016 (+ 434us) server_negotiation.cc:299] Negotiation successful
0928 15:57:43.311243 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 15:57:43.311692 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:43.293523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38340 (local address 127.22.125.252:45153)
0928 15:57:43.293764 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:43.293773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:43.294788 (+ 1015us) server_negotiation.cc:407] Connection header received
0928 15:57:43.295822 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:43.295828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:43.295907 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:43.295998 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:43.297688 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.298698 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.300747 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:43.301385 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:43.301504 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:43.304874 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:43.304879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:43.304883 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:43.304914 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:43.308534 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.309427 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.309434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.309441 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.309577 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:43.309998 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:43.310003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:43.310006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:43.310341 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:43.310575 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:43.311159 (+ 584us) server_negotiation.cc:299] Negotiation successful
0928 15:57:43.311355 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 15:57:44.058642 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.057756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:47123)
0928 15:57:44.058315 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.058465 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":312,"threads_started":1}
W20260928 15:57:44.059046 30485 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:37163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.061508 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.060847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:38093)
0928 15:57:44.061205 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.061305 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":180,"threads_started":1}
W20260928 15:57:44.061901 30204 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:37163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.069132 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.068333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:45793)
0928 15:57:44.068779 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.068910 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":209,"threads_started":1}
W20260928 15:57:44.069460 30344 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:37163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.576706 30561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6991a5163c3e404fb77e798a3f36fee3)
I20260928 15:57:44.576754 30560 raft_consensus.cc:482] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6991a5163c3e404fb77e798a3f36fee3)
I20260928 15:57:44.576980 30561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:44.577051 30560 raft_consensus.cc:504] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:44.577970 30561 leader_election.cc:288] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163), cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252:45153)
I20260928 15:57:44.585001 30560 leader_election.cc:288] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163), 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253:38453)
I20260928 15:57:44.590615 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.578922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35750)
0928 15:57:44.590173 (+ 11251us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.590377 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11136,"threads_started":1}
I20260928 15:57:44.590610 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.578107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35744)
0928 15:57:44.590197 (+ 12090us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.590394 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11988,"threads_started":1}
W20260928 15:57:44.593326 30004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
W20260928 15:57:44.593331 29806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.596992 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.596343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35764)
0928 15:57:44.596521 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.596756 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 15:57:44.597254 30004 leader_election.cc:334] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163): Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.598372 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.597783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35780)
0928 15:57:44.597965 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.598169 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260928 15:57:44.598603 29806 leader_election.cc:334] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163): Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.602397 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.591221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:38453 (local address 127.0.0.1:50994)
0928 15:57:44.591919 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.591942 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:44.592097 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:44.595829 (+ 3732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:44.595834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:44.595872 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:44.596133 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:44.596145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.598916 (+ 2771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.598922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:44.600393 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:44.600406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.601675 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.601681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:44.601881 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:44.601917 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 15:57:44.602014 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 15:57:44.602165 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":1768,"threads_started":1}
I20260928 15:57:44.602661 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.591267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.22.125.253:38453)
0928 15:57:44.595504 (+ 4237us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:44.595509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:44.595527 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:57:44.595584 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:44.595589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:44.595674 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:44.595808 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:44.596364 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.598763 (+ 2399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.600536 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.601510 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.601622 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:44.601697 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:44.602141 (+ 444us) server_negotiation.cc:299] Negotiation successful
0928 15:57:44.602282 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4058,"thread_start_us":3890,"threads_started":1}
I20260928 15:57:44.603031 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.578446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:45153 (local address 127.0.0.1:38346)
0928 15:57:44.590799 (+ 12353us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.590817 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:44.590946 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:44.597266 (+ 6320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:44.597275 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:44.597310 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:44.597812 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:44.597825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.599279 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.599286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:44.600877 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:44.600890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.602171 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.602177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:44.602338 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:44.602376 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 15:57:44.602480 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 15:57:44.602614 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12261,"thread_start_us":12588,"threads_started":1}
I20260928 15:57:44.603263 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.578544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.22.125.252:45153)
0928 15:57:44.596885 (+ 18341us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:44.596891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:44.596908 (+ 17us) server_negotiation.cc:407] Connection header received
0928 15:57:44.596965 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:44.596972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:44.597048 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:44.597937 (+ 889us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:44.598099 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.599150 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.601012 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:44.602014 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:44.602155 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:44.602212 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:44.602695 (+ 483us) server_negotiation.cc:299] Negotiation successful
0928 15:57:44.602867 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18202,"spinlock_wait_cycles":12544,"thread_start_us":18174,"threads_started":1}
I20260928 15:57:44.603263 29831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cebe265c0fea44a194e527f18e02d7d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "03e6b21e733144fd9d68a8309e7361d3" is_pre_election: true
I20260928 15:57:44.603639 29831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cebe265c0fea44a194e527f18e02d7d5 in term 1.
I20260928 15:57:44.603827 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03e6b21e733144fd9d68a8309e7361d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cebe265c0fea44a194e527f18e02d7d5" is_pre_election: true
I20260928 15:57:44.604111 30004 leader_election.cc:302] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03e6b21e733144fd9d68a8309e7361d3, cebe265c0fea44a194e527f18e02d7d5; no voters: 6991a5163c3e404fb77e798a3f36fee3
I20260928 15:57:44.604212 30030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03e6b21e733144fd9d68a8309e7361d3 in term 1.
I20260928 15:57:44.604525 30560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 15:57:44.604656 30560 raft_consensus.cc:482] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6991a5163c3e404fb77e798a3f36fee3)
I20260928 15:57:44.604663 29808 leader_election.cc:302] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03e6b21e733144fd9d68a8309e7361d3, cebe265c0fea44a194e527f18e02d7d5; no voters: 6991a5163c3e404fb77e798a3f36fee3
I20260928 15:57:44.604781 30560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 1 FOLLOWER]: Advancing to term 2
I20260928 15:57:44.604977 30561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 15:57:44.605120 30561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6991a5163c3e404fb77e798a3f36fee3)
I20260928 15:57:44.605245 30561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 1 FOLLOWER]: Advancing to term 2
I20260928 15:57:44.606963 30560 raft_consensus.cc:504] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:44.607307 30561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:44.607622 30560 leader_election.cc:288] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 2 election: Requested vote from peers 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163), 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253:38453)
I20260928 15:57:44.607971 30561 leader_election.cc:288] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 2 election: Requested vote from peers 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163), cebe265c0fea44a194e527f18e02d7d5 (127.22.125.252:45153)
I20260928 15:57:44.608242 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.607642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35788)
0928 15:57:44.607894 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.607993 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260928 15:57:44.608359 29831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cebe265c0fea44a194e527f18e02d7d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "03e6b21e733144fd9d68a8309e7361d3"
I20260928 15:57:44.608543 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.608059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35804)
0928 15:57:44.608250 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.608333 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260928 15:57:44.608471 30030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03e6b21e733144fd9d68a8309e7361d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cebe265c0fea44a194e527f18e02d7d5"
I20260928 15:57:44.608682 29831 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cebe265c0fea44a194e527f18e02d7d5 in current term 2: Already voted for candidate 03e6b21e733144fd9d68a8309e7361d3 in this term.
I20260928 15:57:44.608778 30030 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 03e6b21e733144fd9d68a8309e7361d3 in current term 2: Already voted for candidate cebe265c0fea44a194e527f18e02d7d5 in this term.
I20260928 15:57:44.609745 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.609376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35816)
0928 15:57:44.609541 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.609621 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 15:57:44.609740 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:44.609388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:35822)
0928 15:57:44.609541 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:44.609621 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260928 15:57:44.609908 30004 leader_election.cc:334] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163): Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.610064 30004 leader_election.cc:302] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cebe265c0fea44a194e527f18e02d7d5; no voters: 03e6b21e733144fd9d68a8309e7361d3, 6991a5163c3e404fb77e798a3f36fee3
I20260928 15:57:44.610339 30560 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260928 15:57:44.610503 29806 leader_election.cc:334] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163): Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:44.610677 29806 leader_election.cc:302] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03e6b21e733144fd9d68a8309e7361d3; no voters: 6991a5163c3e404fb77e798a3f36fee3, cebe265c0fea44a194e527f18e02d7d5
I20260928 15:57:44.610929 30561 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 15:57:45.060593 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:45.059731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:34971)
0928 15:57:45.060291 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:45.060388 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":355,"threads_started":1}
I20260928 15:57:45.063302 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:45.062567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:58605)
0928 15:57:45.063021 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:45.063118 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":252,"threads_started":1}
I20260928 15:57:45.070815 30572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:45.070106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:51257)
0928 15:57:45.070568 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:45.070667 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":274,"threads_started":1}
I20260928 15:57:46.062220 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.061421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:37389)
0928 15:57:46.061958 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.062058 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":273,"threads_started":1}
I20260928 15:57:46.065022 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.064280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:52503)
0928 15:57:46.064751 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.064863 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":250,"threads_started":1}
I20260928 15:57:46.072454 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.071611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:59047)
0928 15:57:46.072160 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.072262 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":339,"threads_started":1}
I20260928 15:57:46.434391 30576 raft_consensus.cc:482] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:46.434623 30576 raft_consensus.cc:504] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:46.435225 30576 leader_election.cc:288] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163), 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253:38453)
I20260928 15:57:46.436328 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.435410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36084)
0928 15:57:46.435935 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.436133 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":267,"threads_started":1}
I20260928 15:57:46.436316 29831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cebe265c0fea44a194e527f18e02d7d5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "03e6b21e733144fd9d68a8309e7361d3" is_pre_election: true
I20260928 15:57:46.436601 29831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cebe265c0fea44a194e527f18e02d7d5 in term 2.
I20260928 15:57:46.437071 30004 leader_election.cc:302] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e6b21e733144fd9d68a8309e7361d3, cebe265c0fea44a194e527f18e02d7d5; no voters:
I20260928 15:57:46.437391 30576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260928 15:57:46.437530 30576 raft_consensus.cc:482] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:46.437654 30576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 2 FOLLOWER]: Advancing to term 3
I20260928 15:57:46.437892 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.437457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36090)
0928 15:57:46.437619 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.437731 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 15:57:46.438067 30004 leader_election.cc:334] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163): Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:46.439803 30576 raft_consensus.cc:504] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:46.440336 30576 leader_election.cc:288] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 3 election: Requested vote from peers 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163), 03e6b21e733144fd9d68a8309e7361d3 (127.22.125.253:38453)
I20260928 15:57:46.440841 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.440430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36100)
0928 15:57:46.440615 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.440700 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 15:57:46.440979 29831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cebe265c0fea44a194e527f18e02d7d5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "03e6b21e733144fd9d68a8309e7361d3"
I20260928 15:57:46.441185 29831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 2 FOLLOWER]: Advancing to term 3
I20260928 15:57:46.441970 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.441703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36116)
0928 15:57:46.441818 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.441881 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 15:57:46.442181 30004 leader_election.cc:334] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163): Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
I20260928 15:57:46.443531 29831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cebe265c0fea44a194e527f18e02d7d5 in term 3.
I20260928 15:57:46.443890 30004 leader_election.cc:302] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03e6b21e733144fd9d68a8309e7361d3, cebe265c0fea44a194e527f18e02d7d5; no voters: 6991a5163c3e404fb77e798a3f36fee3
I20260928 15:57:46.444170 30576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 15:57:46.444581 30576 raft_consensus.cc:686] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [term 3 LEADER]: Becoming Leader. State: Replica: cebe265c0fea44a194e527f18e02d7d5, State: Running, Role: LEADER
I20260928 15:57:46.444958 30576 consensus_queue.cc:237] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } }
I20260928 15:57:46.446561 30579 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cebe265c0fea44a194e527f18e02d7d5. Latest consensus state: current_term: 3 leader_uuid: "cebe265c0fea44a194e527f18e02d7d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:46.446841 30579 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:46.447463 30581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:46.451334 30581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:46.452243 30581 catalog_manager.cc:1259] Loaded cluster ID: 6aa39b3fca66401e9cb6881c23cd4302
I20260928 15:57:46.452338 30581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:46.453469 30581 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:46.454288 30581 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Loaded TSK: 0
I20260928 15:57:46.455184 30581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 15:57:46.504768 30019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:46.511641 30019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:46.515877 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.515428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36146)
0928 15:57:46.515634 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.515730 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260928 15:57:46.516232 29831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [term 3 FOLLOWER]: Refusing update from remote peer cebe265c0fea44a194e527f18e02d7d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260928 15:57:46.516971 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.516519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36154)
0928 15:57:46.516700 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.516782 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 15:57:46.516952 30579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260928 15:57:46.517331 30004 consensus_peers.cc:487] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 -> Peer 6991a5163c3e404fb77e798a3f36fee3 (127.22.125.254:37163): Couldn't send request to peer 6991a5163c3e404fb77e798a3f36fee3. Status: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 15:57:46.519577 30583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cebe265c0fea44a194e527f18e02d7d5. Latest consensus state: current_term: 3 leader_uuid: "cebe265c0fea44a194e527f18e02d7d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:46.519891 30583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:46.526221 30576 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cebe265c0fea44a194e527f18e02d7d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:46.526620 30576 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5 [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:46.527230 30583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cebe265c0fea44a194e527f18e02d7d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6991a5163c3e404fb77e798a3f36fee3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37163 } } peers { permanent_uuid: "03e6b21e733144fd9d68a8309e7361d3" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 38453 } attrs { promote: false } } peers { permanent_uuid: "cebe265c0fea44a194e527f18e02d7d5" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 45153 } attrs { promote: false } } }
I20260928 15:57:46.527542 30583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03e6b21e733144fd9d68a8309e7361d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 15:57:46.547225 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.536781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.22.125.195:44653)
0928 15:57:46.537293 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:46.537303 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:46.540933 (+ 3630us) server_negotiation.cc:407] Connection header received
0928 15:57:46.541228 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:46.541235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:46.541320 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:46.541416 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:46.542322 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.543282 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.545201 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.546072 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.546276 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.546353 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:46.546727 (+ 374us) server_negotiation.cc:299] Negotiation successful
0928 15:57:46.546892 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":277,"threads_started":1}
I20260928 15:57:46.548238 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.536293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:42437 (local address 127.0.0.1:45076)
0928 15:57:46.536543 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.536570 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:46.536697 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:46.540919 (+ 4222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:46.540923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:46.540956 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:46.541325 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.541337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.542957 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.542962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.545260 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.545274 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.547387 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.547395 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.547565 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.547596 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 15:57:46.547680 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 15:57:46.547837 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":89}
I20260928 15:57:46.548465 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.536382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.22.125.193:42437)
0928 15:57:46.540541 (+ 4159us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:46.540549 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:46.540572 (+ 23us) server_negotiation.cc:407] Connection header received
0928 15:57:46.540652 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:46.540659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:46.540749 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:46.541480 (+ 731us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:46.541678 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.542816 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.545438 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.547197 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.547369 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.547454 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:46.547966 (+ 512us) server_negotiation.cc:299] Negotiation successful
0928 15:57:46.548129 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4022,"thread_start_us":3886,"threads_started":1}
I20260928 15:57:46.548308 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.536381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44653 (local address 127.0.0.1:45908)
0928 15:57:46.540833 (+ 4452us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.540851 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:46.541044 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:46.541704 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:46.541708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:46.541734 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:46.542085 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.542098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.543423 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.543426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.545050 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.545062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.546282 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.546288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.546492 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.546525 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 15:57:46.547657 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0928 15:57:46.547759 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4300,"mutex_wait_us":70,"thread_start_us":4135,"threads_started":1}
I20260928 15:57:46.548718 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.537260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.22.125.194:34925)
0928 15:57:46.537904 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:46.537913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:46.541305 (+ 3392us) server_negotiation.cc:407] Connection header received
0928 15:57:46.541584 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:46.541589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:46.541680 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:46.541781 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:46.542757 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.543875 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.546399 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.547435 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.547558 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.547629 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:46.548241 (+ 612us) server_negotiation.cc:299] Negotiation successful
0928 15:57:46.548395 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":317,"threads_started":1}
I20260928 15:57:46.549525 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.536776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:34925 (local address 127.0.0.1:47092)
0928 15:57:46.541197 (+ 4421us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.541211 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:46.541388 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:46.542045 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:46.542049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:46.542081 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:46.542564 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.542578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.544092 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.544097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.546212 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.546231 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.547819 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.547824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.548005 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.548070 (+ 65us) client_negotiation.cc:769] Sending connection context
0928 15:57:46.549052 (+ 982us) client_negotiation.cc:240] Negotiation successful
0928 15:57:46.549172 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4319,"mutex_wait_us":3,"thread_start_us":4155,"threads_started":1}
I20260928 15:57:46.549413 30419 tablet_service.cc:1467] Processing CreateTablet for tablet 262a42908e884d4596ff81a8ffafe5ae (DEFAULT_TABLE table=test-table [id=ee6924845a6943ddb4ce06d9efe5f45b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:46.549923 30419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 262a42908e884d4596ff81a8ffafe5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:46.550006 30138 tablet_service.cc:1467] Processing CreateTablet for tablet 262a42908e884d4596ff81a8ffafe5ae (DEFAULT_TABLE table=test-table [id=ee6924845a6943ddb4ce06d9efe5f45b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:46.550343 30278 tablet_service.cc:1467] Processing CreateTablet for tablet 262a42908e884d4596ff81a8ffafe5ae (DEFAULT_TABLE table=test-table [id=ee6924845a6943ddb4ce06d9efe5f45b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:46.550447 30138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 262a42908e884d4596ff81a8ffafe5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:46.550729 30278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 262a42908e884d4596ff81a8ffafe5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:46.555874 30594 tablet_bootstrap.cc:492] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Bootstrap starting.
I20260928 15:57:46.556897 30595 tablet_bootstrap.cc:492] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: Bootstrap starting.
I20260928 15:57:46.557570 30596 tablet_bootstrap.cc:492] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: Bootstrap starting.
I20260928 15:57:46.558643 30595 tablet_bootstrap.cc:654] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:46.558969 30594 tablet_bootstrap.cc:654] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:46.559522 30596 tablet_bootstrap.cc:654] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:46.560626 30595 tablet_bootstrap.cc:492] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: No bootstrap required, opened a new log
I20260928 15:57:46.560776 30595 ts_tablet_manager.cc:1397] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 15:57:46.561671 30596 tablet_bootstrap.cc:492] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: No bootstrap required, opened a new log
I20260928 15:57:46.561589 30595 raft_consensus.cc:348] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.561796 30595 raft_consensus.cc:374] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:46.561825 30596 ts_tablet_manager.cc:1397] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260928 15:57:46.561936 30595 raft_consensus.cc:729] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3db5ab25c94440a80c3ab45878cd12a, State: Initialized, Role: FOLLOWER
I20260928 15:57:46.562027 30594 tablet_bootstrap.cc:492] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: No bootstrap required, opened a new log
I20260928 15:57:46.562152 30594 ts_tablet_manager.cc:1397] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260928 15:57:46.562196 30595 consensus_queue.cc:260] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.563088 30595 ts_tablet_manager.cc:1428] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:46.562714 30596 raft_consensus.cc:348] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.563535 30594 raft_consensus.cc:348] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.563668 30596 raft_consensus.cc:374] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:46.563779 30596 raft_consensus.cc:729] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c37d8527da493287e4d0702b9600fc, State: Initialized, Role: FOLLOWER
I20260928 15:57:46.563800 30594 raft_consensus.cc:374] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:46.564107 30596 consensus_queue.cc:260] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.564517 30594 raft_consensus.cc:729] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718611922ca04e478eef8495c05fb4f7, State: Initialized, Role: FOLLOWER
I20260928 15:57:46.564853 30594 consensus_queue.cc:260] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.566797 30484 heartbeater.cc:503] Master 127.22.125.252:45153 was elected leader, sending a full tablet report...
I20260928 15:57:46.569120 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.563786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:51651)
0928 15:57:46.568806 (+ 5020us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.568913 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4887,"thread_start_us":4819,"threads_started":1}
I20260928 15:57:46.569823 30594 ts_tablet_manager.cc:1428] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260928 15:57:46.570385 30596 ts_tablet_manager.cc:1428] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260928 15:57:46.571483 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.570320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:59285)
0928 15:57:46.571235 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.571329 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":804}
I20260928 15:57:46.571540 30343 heartbeater.cc:503] Master 127.22.125.252:45153 was elected leader, sending a full tablet report...
I20260928 15:57:46.572836 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.572101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:60743)
0928 15:57:46.572562 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.572647 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":265,"threads_started":1}
I20260928 15:57:46.573040 30203 heartbeater.cc:503] Master 127.22.125.252:45153 was elected leader, sending a full tablet report...
I20260928 15:57:46.574075 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.573690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:48847)
0928 15:57:46.573854 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.573937 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 15:57:46.575368 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.574991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:60457)
0928 15:57:46.575131 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.575215 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 15:57:46.819389 30346 tablet.cc:2369] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:57:46.870821 30603 raft_consensus.cc:482] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:46.871057 30603 raft_consensus.cc:504] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.872589 30603 leader_election.cc:288] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437), d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:46.879446 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.873073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:49091 (local address 127.22.125.195:44653)
0928 15:57:46.873259 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:46.873264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:46.873290 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:57:46.873383 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:46.873387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:46.873445 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:46.873534 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:46.874250 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.875229 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.877592 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.878474 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.878645 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.878772 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:46.879053 (+ 281us) server_negotiation.cc:299] Negotiation successful
0928 15:57:46.879194 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 15:57:46.879475 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.872916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44653 (local address 127.22.125.194:49091)
0928 15:57:46.873097 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.873128 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:46.873262 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:46.873600 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:46.873608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:46.873653 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:46.874031 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.874041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.875357 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.875362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.877444 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.877456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.878659 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.878665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.878852 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.878903 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 15:57:46.878991 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 15:57:46.879137 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 15:57:46.879637 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.873085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:55635 (local address 127.22.125.193:42437)
0928 15:57:46.873277 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:46.873284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:46.874107 (+ 823us) server_negotiation.cc:407] Connection header received
0928 15:57:46.874243 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:46.874247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:46.874305 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:46.874377 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:46.874916 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.875826 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.877819 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.878664 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.878816 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.878885 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:46.879227 (+ 342us) server_negotiation.cc:299] Negotiation successful
0928 15:57:46.879374 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 15:57:46.879940 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.872916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:42437 (local address 127.22.125.194:55635)
0928 15:57:46.873985 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.874001 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:46.874104 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:46.874418 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:46.874422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:46.874445 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:46.874758 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.874768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.875956 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.875960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.877688 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:46.877702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.878898 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.878903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:46.879082 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.879114 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 15:57:46.879199 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 15:57:46.879317 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":435,"thread_start_us":331,"threads_started":1}
I20260928 15:57:46.880445 30439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262a42908e884d4596ff81a8ffafe5ae" candidate_uuid: "718611922ca04e478eef8495c05fb4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3db5ab25c94440a80c3ab45878cd12a" is_pre_election: true
I20260928 15:57:46.880745 30439 raft_consensus.cc:2454] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 718611922ca04e478eef8495c05fb4f7 in term 0.
I20260928 15:57:46.880851 30158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262a42908e884d4596ff81a8ffafe5ae" candidate_uuid: "718611922ca04e478eef8495c05fb4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c37d8527da493287e4d0702b9600fc" is_pre_election: true
I20260928 15:57:46.881229 30158 raft_consensus.cc:2454] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 718611922ca04e478eef8495c05fb4f7 in term 0.
I20260928 15:57:46.881348 30231 leader_election.cc:302] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 718611922ca04e478eef8495c05fb4f7, d3db5ab25c94440a80c3ab45878cd12a; no voters:
I20260928 15:57:46.881793 30603 raft_consensus.cc:2781] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:46.881918 30603 raft_consensus.cc:482] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:46.882019 30603 raft_consensus.cc:3037] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:46.884534 30603 raft_consensus.cc:504] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.885174 30603 leader_election.cc:288] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [CANDIDATE]: Term 1 election: Requested vote from peers f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437), d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653)
I20260928 15:57:46.885563 30158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262a42908e884d4596ff81a8ffafe5ae" candidate_uuid: "718611922ca04e478eef8495c05fb4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c37d8527da493287e4d0702b9600fc"
I20260928 15:57:46.885778 30158 raft_consensus.cc:3037] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:46.885730 30439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262a42908e884d4596ff81a8ffafe5ae" candidate_uuid: "718611922ca04e478eef8495c05fb4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3db5ab25c94440a80c3ab45878cd12a"
I20260928 15:57:46.885936 30439 raft_consensus.cc:3037] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:46.888875 30439 raft_consensus.cc:2454] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 718611922ca04e478eef8495c05fb4f7 in term 1.
I20260928 15:57:46.888881 30158 raft_consensus.cc:2454] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 718611922ca04e478eef8495c05fb4f7 in term 1.
I20260928 15:57:46.889307 30231 leader_election.cc:302] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 718611922ca04e478eef8495c05fb4f7, d3db5ab25c94440a80c3ab45878cd12a, f4c37d8527da493287e4d0702b9600fc; no voters:
I20260928 15:57:46.889668 30603 raft_consensus.cc:2781] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:46.890133 30603 raft_consensus.cc:686] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 1 LEADER]: Becoming Leader. State: Replica: 718611922ca04e478eef8495c05fb4f7, State: Running, Role: LEADER
I20260928 15:57:46.890864 30603 consensus_queue.cc:237] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } }
I20260928 15:57:46.891184 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.890719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:53619)
0928 15:57:46.890892 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.890994 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 15:57:46.897823 30019 catalog_manager.cc:5495] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 718611922ca04e478eef8495c05fb4f7 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "718611922ca04e478eef8495c05fb4f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "718611922ca04e478eef8495c05fb4f7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34925 } health_report { overall_health: HEALTHY } } }
I20260928 15:57:46.917172 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.916735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36168)
0928 15:57:46.916924 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.917025 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 15:57:46.918181 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.917870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36178)
0928 15:57:46.917994 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:46.918064 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 15:57:46.997062 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:46.992318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47100 (local address 127.22.125.194:34925)
0928 15:57:46.992493 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:46.992497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:46.992515 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:57:46.992698 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:46.992702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:46.992778 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:46.992908 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:46.993209 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.993901 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.995082 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:46.995467 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:46.995554 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:46.995912 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:46.996506 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:46.996613 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:46.996696 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 15:57:46.996756 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 15:57:47.001385 30439 raft_consensus.cc:1261] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Refusing update from remote peer 718611922ca04e478eef8495c05fb4f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:47.001641 30158 raft_consensus.cc:1261] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 1 FOLLOWER]: Refusing update from remote peer 718611922ca04e478eef8495c05fb4f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:47.002144 30606 consensus_queue.cc:1035] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3db5ab25c94440a80c3ab45878cd12a" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:47.002533 30603 consensus_queue.cc:1035] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c37d8527da493287e4d0702b9600fc" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:47.005714 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.005221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:39691)
0928 15:57:47.005469 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:47.005566 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260928 15:57:47.006524 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.006068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:32873)
0928 15:57:47.006235 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:47.006324 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 15:57:47.011384 30609 mvcc.cc:205] Tried to move back new op lower bound from 7334342930429616128 to 7334342929989349376. Current Snapshot: MvccSnapshot[applied={T|T < 7334342930429616128}]
I20260928 15:57:47.013581 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.013077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.194:54027)
0928 15:57:47.013289 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:47.013408 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260928 15:57:47.014845 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.014382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.193:57919)
0928 15:57:47.014581 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:47.014688 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260928 15:57:47.025790 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.025061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.22.125.195:48179)
0928 15:57:47.025333 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:47.025439 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260928 15:57:47.025753 30019 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:47.026014 30019 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:47.032900 30019 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cebe265c0fea44a194e527f18e02d7d5: Sending DeleteTablet for 3 replicas of tablet 262a42908e884d4596ff81a8ffafe5ae
I20260928 15:57:47.034207 30419 tablet_service.cc:1514] Processing DeleteTablet for tablet 262a42908e884d4596ff81a8ffafe5ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:47 UTC) from {username='oryx', principal='oryx/127.22.125.252@KRBTEST.COM'} at 127.0.0.1:45908
I20260928 15:57:47.034333 30418 tablet_service.cc:1514] Processing DeleteTablet for tablet 262a42908e884d4596ff81a8ffafe5ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:47 UTC) from {username='oryx', principal='oryx/127.22.125.252@KRBTEST.COM'} at 127.0.0.1:45908
I20260928 15:57:47.035276 30138 tablet_service.cc:1514] Processing DeleteTablet for tablet 262a42908e884d4596ff81a8ffafe5ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:47 UTC) from {username='oryx', principal='oryx/127.22.125.252@KRBTEST.COM'} at 127.0.0.1:45076
I20260928 15:57:47.036901 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29993
I20260928 15:57:47.037411 30278 tablet_service.cc:1514] Processing DeleteTablet for tablet 262a42908e884d4596ff81a8ffafe5ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:47 UTC) from {username='oryx', principal='oryx/127.22.125.252@KRBTEST.COM'} at 127.0.0.1:47092
I20260928 15:57:47.045387 30619 tablet_replica.cc:330] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: stopping tablet replica
I20260928 15:57:47.046042 30619 raft_consensus.cc:2229] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:47.046738 30619 raft_consensus.cc:2258] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:47.048501 30619 ts_tablet_manager.cc:1905] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:47.057955 30620 tablet_replica.cc:330] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: stopping tablet replica
I20260928 15:57:47.058385 30620 raft_consensus.cc:2229] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:47.059087 30621 tablet_replica.cc:330] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: stopping tablet replica
I20260928 15:57:47.059530 30620 raft_consensus.cc:2258] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:47.060329 30621 raft_consensus.cc:2229] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:47.061249 30620 ts_tablet_manager.cc:1905] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:47.062733 30621 raft_consensus.cc:2258] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:47.065675 30621 ts_tablet_manager.cc:1905] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:47.073342 30621 ts_tablet_manager.cc:1918] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:47.074121 30621 log.cc:1196] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/262a42908e884d4596ff81a8ffafe5ae
I20260928 15:57:47.074751 30621 ts_tablet_manager.cc:1939] T 262a42908e884d4596ff81a8ffafe5ae P 718611922ca04e478eef8495c05fb4f7: Deleting consensus metadata
I20260928 15:57:47.075810 30620 ts_tablet_manager.cc:1918] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:47.075947 30620 log.cc:1196] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/262a42908e884d4596ff81a8ffafe5ae
I20260928 15:57:47.076184 30619 ts_tablet_manager.cc:1918] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:47.076383 30619 log.cc:1196] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/262a42908e884d4596ff81a8ffafe5ae
I20260928 15:57:47.076573 30620 ts_tablet_manager.cc:1939] T 262a42908e884d4596ff81a8ffafe5ae P f4c37d8527da493287e4d0702b9600fc: Deleting consensus metadata
I20260928 15:57:47.076997 30619 ts_tablet_manager.cc:1939] T 262a42908e884d4596ff81a8ffafe5ae P d3db5ab25c94440a80c3ab45878cd12a: Deleting consensus metadata
I20260928 15:57:47.077813 30004 catalog_manager.cc:4841] TS f4c37d8527da493287e4d0702b9600fc (127.22.125.193:42437): tablet 262a42908e884d4596ff81a8ffafe5ae (table test-table [id=ee6924845a6943ddb4ce06d9efe5f45b]) successfully deleted
I20260928 15:57:47.078923 30003 catalog_manager.cc:4841] TS d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653): tablet 262a42908e884d4596ff81a8ffafe5ae (table test-table [id=ee6924845a6943ddb4ce06d9efe5f45b]) successfully deleted
W20260928 15:57:47.079358 30003 catalog_manager.cc:4808] TS d3db5ab25c94440a80c3ab45878cd12a (127.22.125.195:44653): delete failed for tablet 262a42908e884d4596ff81a8ffafe5ae because the tablet was not found. No further retry: Not found: Tablet not found: 262a42908e884d4596ff81a8ffafe5ae
I20260928 15:57:47.081243 30004 catalog_manager.cc:4841] TS 718611922ca04e478eef8495c05fb4f7 (127.22.125.194:34925): tablet 262a42908e884d4596ff81a8ffafe5ae (table test-table [id=ee6924845a6943ddb4ce06d9efe5f45b]) successfully deleted
I20260928 15:57:47.082861 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.044257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45080 (local address 127.22.125.193:42437)
0928 15:57:47.044462 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:47.044469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:47.064989 (+ 20520us) server_negotiation.cc:407] Connection header received
0928 15:57:47.065284 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:47.065289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:47.065412 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:47.065493 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:47.067629 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:47.068706 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:47.070504 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:47.071144 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:47.071244 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:47.074949 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:47.074968 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:47.074991 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:47.075054 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:47.079353 (+ 4299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:47.080373 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:47.080384 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:47.080388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:47.080548 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:47.081032 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:47.081040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:47.081046 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:47.081521 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:47.081758 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:47.082249 (+ 491us) server_negotiation.cc:299] Negotiation successful
0928 15:57:47.082477 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 15:57:47.083614 30198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:47.205615 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29993
I20260928 15:57:47.228103 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 30215
I20260928 15:57:47.243824 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.228931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47104 (local address 127.22.125.194:34925)
0928 15:57:47.229098 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:47.229102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:47.229140 (+ 38us) server_negotiation.cc:407] Connection header received
0928 15:57:47.229352 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:47.229355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:47.229441 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:47.229519 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:47.231291 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:47.232326 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:47.234134 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:47.234578 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:47.234666 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:47.237902 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:47.237916 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:47.237931 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:47.237963 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:47.241193 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:47.241917 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:47.241923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:47.241925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:47.242071 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:47.242474 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:47.242477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:47.242479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:47.242767 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:47.242982 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:47.243303 (+ 321us) server_negotiation.cc:299] Negotiation successful
0928 15:57:47.243455 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 15:57:47.244612 30338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:47.331678 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.331187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36194)
0928 15:57:47.331377 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:47.331505 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260928 15:57:47.332793 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.332391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37163 (local address 127.0.0.1:36204)
0928 15:57:47.332551 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:47.332633 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 15:57:47.365975 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 30215
I20260928 15:57:47.385843 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 30356
I20260928 15:57:47.398453 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:47.386541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.22.125.195:44653)
0928 15:57:47.386977 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:47.386981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:47.386996 (+ 15us) server_negotiation.cc:407] Connection header received
0928 15:57:47.387053 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:47.387056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:47.387114 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:47.387207 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:47.388581 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:47.389270 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:47.390418 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:47.390823 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:47.390920 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:47.393410 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:47.393424 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:47.393439 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:47.393478 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:47.396083 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:47.396671 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:47.396677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:47.396680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:47.396811 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:47.397121 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:47.397124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:47.397126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:47.397460 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:47.397670 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:47.397935 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 15:57:47.398092 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":243,"threads_started":1}
I20260928 15:57:47.399087 30479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:47.493495 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 30356
I20260928 15:57:47.510864 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29675
I20260928 15:57:47.511942 29851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:47.631009 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29675
I20260928 15:57:47.651235 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 29775
I20260928 15:57:47.652388 30050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:47.775718 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 29775
2026-09-28T15:57:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11574 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:47 dist-test-slave-9cqd krb5kdc[30631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:47 dist-test-slave-9cqd krb5kdc[30631](info): set up 2 sockets
Sep 28 15:57:47 dist-test-slave-9cqd krb5kdc[30631](info): commencing operation
krb5kdc: starting...
W20260928 15:57:48.870831 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:49 dist-test-slave-9cqd krb5kdc[30631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab.
Entry for principal oryx/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab.
I20260928 15:57:49.080744 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44091
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab
--principal=oryx/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:37595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:44091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:49.399529 30644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:49.399870 30644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:49.400115 30644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:49.407407 30644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:49.407548 30644 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 15:57:49.407683 30644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:49.407796 30644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:49.407894 30644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:49.407979 30644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:49.417850 30644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:44091
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.254.keytab
--principal=oryx/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:49.419157 30644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:49.420534 30644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:49.432403 30652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:49.432495 30650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:49.432502 30649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:49.432762 30644 server_base.cc:1034] running on GCE node
I20260928 15:57:49.433856 30644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:49.435137 30644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:49.436378 30644 hybrid_clock.cc:648] HybridClock initialized: now 1790611069436311 us; error 38 us; skew 500 ppm
I20260928 15:57:49.436697 30644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:49 dist-test-slave-9cqd krb5kdc[30631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611069, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:49.446229 30644 init.cc:377] Logged in from keytab as oryx/127.22.125.254@KRBTEST.COM (short username oryx)
I20260928 15:57:49.447762 30644 webserver.cc:460] Webserver started at http://127.22.125.254:40205/ using document root <none> and password file <none>
I20260928 15:57:49.448505 30644 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:49.448594 30644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:49.448868 30644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:49.451726 30644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "52d8a9b460a04157b1ab83500c1a2c8d"
format_stamp: "Formatted at 2026-09-28 15:57:49 on dist-test-slave-9cqd"
server_key: "eb75987259e9d51def04f01c3ac99958"
server_key_iv: "f58840877bc7771806502ec9d2d62768"
server_key_version: "encryptionkey@0"
I20260928 15:57:49.452538 30644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "52d8a9b460a04157b1ab83500c1a2c8d"
format_stamp: "Formatted at 2026-09-28 15:57:49 on dist-test-slave-9cqd"
server_key: "eb75987259e9d51def04f01c3ac99958"
server_key_iv: "f58840877bc7771806502ec9d2d62768"
server_key_version: "encryptionkey@0"
I20260928 15:57:49.457396 30644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260928 15:57:49.460656 30659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:49.461926 30644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:49.462087 30644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "52d8a9b460a04157b1ab83500c1a2c8d"
format_stamp: "Formatted at 2026-09-28 15:57:49 on dist-test-slave-9cqd"
server_key: "eb75987259e9d51def04f01c3ac99958"
server_key_iv: "f58840877bc7771806502ec9d2d62768"
server_key_version: "encryptionkey@0"
I20260928 15:57:49.462275 30644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:49.496973 30644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:49.501027 30644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:49.501302 30644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:49.522953 30644 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:44091
I20260928 15:57:49.522974 30720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:44091 every 8 connection(s)
I20260928 15:57:49.524351 30644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:49.528003 30721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:49.531780 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 30644
I20260928 15:57:49.532020 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:49.532879 23031 external_mini_cluster.cc:1428] Setting key c15fb25873c3ff37c52eda3610e3b372
I20260928 15:57:49.541540 30721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Bootstrap starting.
I20260928 15:57:49.545215 30721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:49.546584 30721 log.cc:824] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Log is configured to *not* fsync() on all Append() calls
Sep 28 15:57:49 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:49.549787 30721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: No bootstrap required, opened a new log
I20260928 15:57:49.555712 30724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:49.534910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.22.125.254:44091)
0928 15:57:49.535685 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:49.535696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:49.535735 (+ 39us) server_negotiation.cc:407] Connection header received
0928 15:57:49.537403 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:49.537424 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:49.537803 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:49.538123 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:49.539863 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:49.541165 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:49.542820 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:49.543474 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:49.543593 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:49.548610 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:49.548618 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:49.548632 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:49.548665 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:49.551741 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:49.552517 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:49.552524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:49.552529 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:49.552697 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:49.553080 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:49.553086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:49.553089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:49.553616 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:49.553942 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:49.554338 (+ 396us) server_negotiation.cc:299] Negotiation successful
0928 15:57:49.554738 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":390,"threads_started":1}
I20260928 15:57:49.558229 30721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44091 } }
I20260928 15:57:49.558717 30721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:49.558821 30721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d8a9b460a04157b1ab83500c1a2c8d, State: Initialized, Role: FOLLOWER
I20260928 15:57:49.559324 30721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44091 } }
I20260928 15:57:49.559527 30721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:49.559623 30721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:49.559732 30721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:49.562240 30721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44091 } }
I20260928 15:57:49.562646 30721 leader_election.cc:302] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52d8a9b460a04157b1ab83500c1a2c8d; no voters:
I20260928 15:57:49.563207 30721 leader_election.cc:288] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:49.563289 30726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:49.564087 30726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [term 1 LEADER]: Becoming Leader. State: Replica: 52d8a9b460a04157b1ab83500c1a2c8d, State: Running, Role: LEADER
I20260928 15:57:49.564836 30726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44091 } }
I20260928 15:57:49.565460 30721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:49.567799 30727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44091 } } }
I20260928 15:57:49.568279 30727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:49.567826 30728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52d8a9b460a04157b1ab83500c1a2c8d. Latest consensus state: current_term: 1 leader_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d8a9b460a04157b1ab83500c1a2c8d" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44091 } } }
I20260928 15:57:49.568826 30728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:49.572801 30735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:49.577996 30735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:49.586302 30735 catalog_manager.cc:1347] Generated new cluster ID: c68c7db0a9274767955986920d1b4f83
I20260928 15:57:49.586402 30735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:49.598506 30735 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:49.599757 30735 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:49.613529 30735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Generated new TSK 0
I20260928 15:57:49.614672 30735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab.
Entry for principal oryx/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab.
I20260928 15:57:49.677737 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44091
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab
--principal=oryx/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:37595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:50.000183 30747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:50.000500 30747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:50.000689 30747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:50.007889 30747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:50.008028 30747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:50.008188 30747 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:57:50.008320 30747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:50.017535 30747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab
--principal=oryx/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:50.018740 30747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:50.020089 30747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:50.032982 30753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:50.032948 30755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:50.033102 30752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:50.034204 30747 server_base.cc:1034] running on GCE node
I20260928 15:57:50.034808 30747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:50.035751 30747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:50.036986 30747 hybrid_clock.cc:648] HybridClock initialized: now 1790611070036882 us; error 82 us; skew 500 ppm
I20260928 15:57:50.037279 30747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:50 dist-test-slave-9cqd krb5kdc[30631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611070, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:50.040468 30747 init.cc:377] Logged in from keytab as oryx/127.22.125.193@KRBTEST.COM (short username oryx)
I20260928 15:57:50.041850 30747 webserver.cc:460] Webserver started at http://127.22.125.193:42807/ using document root <none> and password file <none>
I20260928 15:57:50.042559 30747 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:50.042694 30747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:50.043023 30747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:50.045881 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "7c14e55f7ad04080b2531520cb4225c5"
format_stamp: "Formatted at 2026-09-28 15:57:50 on dist-test-slave-9cqd"
server_key: "210f2c71d0cd1f6baf599b9ae10fdeaa"
server_key_iv: "c5db4f412f61481115fcc9520d87ab33"
server_key_version: "encryptionkey@0"
I20260928 15:57:50.046684 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "7c14e55f7ad04080b2531520cb4225c5"
format_stamp: "Formatted at 2026-09-28 15:57:50 on dist-test-slave-9cqd"
server_key: "210f2c71d0cd1f6baf599b9ae10fdeaa"
server_key_iv: "c5db4f412f61481115fcc9520d87ab33"
server_key_version: "encryptionkey@0"
I20260928 15:57:50.051673 30747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 15:57:50.055015 30762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:50.056370 30747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 15:57:50.056499 30747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "7c14e55f7ad04080b2531520cb4225c5"
format_stamp: "Formatted at 2026-09-28 15:57:50 on dist-test-slave-9cqd"
server_key: "210f2c71d0cd1f6baf599b9ae10fdeaa"
server_key_iv: "c5db4f412f61481115fcc9520d87ab33"
server_key_version: "encryptionkey@0"
I20260928 15:57:50.056670 30747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:50.074383 30747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:50.078649 30747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:50.078931 30747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:50.081632 30747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:50.081771 30747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:50.081928 30747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:50.082017 30747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:57:50 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611070, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:50.108312 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.081983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:54845 (local address 127.22.125.254:44091)
0928 15:57:50.082440 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:50.082446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:50.083608 (+ 1162us) server_negotiation.cc:407] Connection header received
0928 15:57:50.085867 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:50.085875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:50.085989 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:50.086134 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:50.088389 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.089715 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.092285 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.092691 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.092823 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.098274 (+ 5451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:50.098285 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:50.098293 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:50.098342 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:50.102111 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.103386 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.103394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.103398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.103564 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.104258 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.104265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.104269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.104693 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:50.104860 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:50.107740 (+ 2880us) server_negotiation.cc:299] Negotiation successful
0928 15:57:50.107958 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":234,"threads_started":1}
I20260928 15:57:50.108606 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.082548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44091 (local address 127.22.125.193:54845)
0928 15:57:50.083422 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:50.083464 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:50.085540 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:50.086398 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:50.086411 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:50.087202 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:50.088127 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.088159 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.089928 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.089936 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.092137 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.092149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.092958 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.092969 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.093248 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.094128 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:50.094161 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:50.097984 (+ 3823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:50.102280 (+ 4296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.102297 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.102324 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.103183 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.103708 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.103713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.103718 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.104079 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.104942 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:50.104958 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:50.105633 (+ 675us) client_negotiation.cc:769] Sending connection context
0928 15:57:50.106030 (+ 397us) client_negotiation.cc:240] Negotiation successful
0928 15:57:50.106774 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":350,"threads_started":1}
I20260928 15:57:50.124076 30747 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:39099
I20260928 15:57:50.124101 30882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:39099 every 8 connection(s)
I20260928 15:57:50.125299 30747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:50.127708 30883 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44091
I20260928 15:57:50.127933 30883 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:50.128613 30883 heartbeater.cc:511] Master 127.22.125.254:44091 requested a full tablet report, sending...
I20260928 15:57:50.130617 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 30747
I20260928 15:57:50.130805 30676 ts_manager.cc:194] Registered new tserver with Master: 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099)
I20260928 15:57:50.130880 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:50.131670 23031 external_mini_cluster.cc:1428] Setting key 0b25065bfae735418573b1b0cb25f480
I20260928 15:57:50.132969 30676 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:54845
I20260928 15:57:50.133512 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.115930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:58393 (local address 127.22.125.254:44091)
0928 15:57:50.116168 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:50.116177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:50.120708 (+ 4531us) server_negotiation.cc:407] Connection header received
0928 15:57:50.120878 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:50.120887 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:50.120949 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:50.121029 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:50.122561 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.123456 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.124964 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.125397 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.125538 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.127740 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:50.127746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:50.127757 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:50.127811 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:50.130642 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.131407 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.131414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.131418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.131569 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.131948 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.131953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.131957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.132305 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:50.132497 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:50.132964 (+ 467us) server_negotiation.cc:299] Negotiation successful
0928 15:57:50.133188 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 15:57:50.133555 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.115926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44091 (local address 127.22.125.193:58393)
0928 15:57:50.120595 (+ 4669us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:50.120615 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:50.120764 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:50.121166 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:50.121173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:50.121919 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:50.122403 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.122420 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.123594 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.123600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.124816 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.124828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.125587 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.125599 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.125764 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.126439 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:50.126457 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:50.127542 (+ 1085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:50.130852 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.130866 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.130869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.131283 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.131681 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.131686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.131689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.131838 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.132451 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:50.132457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:50.132668 (+ 211us) client_negotiation.cc:769] Sending connection context
0928 15:57:50.132827 (+ 159us) client_negotiation.cc:240] Negotiation successful
0928 15:57:50.133131 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4531,"thread_start_us":4358,"threads_started":1}
WARNING: no policy specified for oryx/127.22.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab.
Entry for principal oryx/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab.
I20260928 15:57:50.177034 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44091
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab
--principal=oryx/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:37595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:50.492646 30887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:50.492910 30887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:50.493005 30887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:50.500109 30887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:50.500211 30887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:50.500293 30887 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:57:50.500379 30887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:50.509987 30887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab
--principal=oryx/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:50.511257 30887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:50.512633 30887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:50.525946 30893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:50.526028 30892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:50.526942 30895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:50.532382 30887 server_base.cc:1034] running on GCE node
I20260928 15:57:50.533027 30887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:50.533969 30887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:50.535216 30887 hybrid_clock.cc:648] HybridClock initialized: now 1790611070535151 us; error 40 us; skew 500 ppm
I20260928 15:57:50.535527 30887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:50 dist-test-slave-9cqd krb5kdc[30631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611070, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:50.538738 30887 init.cc:377] Logged in from keytab as oryx/127.22.125.194@KRBTEST.COM (short username oryx)
I20260928 15:57:50.540290 30887 webserver.cc:460] Webserver started at http://127.22.125.194:36707/ using document root <none> and password file <none>
I20260928 15:57:50.540977 30887 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:50.541105 30887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:50.541419 30887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:50.544301 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "114d760b6de344ae8c66ae12ce58ec91"
format_stamp: "Formatted at 2026-09-28 15:57:50 on dist-test-slave-9cqd"
server_key: "d0da9d9d7aa56220907fca8722ebc996"
server_key_iv: "b77ce963e596aba8b53c8d0c01edfb63"
server_key_version: "encryptionkey@0"
I20260928 15:57:50.545084 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "114d760b6de344ae8c66ae12ce58ec91"
format_stamp: "Formatted at 2026-09-28 15:57:50 on dist-test-slave-9cqd"
server_key: "d0da9d9d7aa56220907fca8722ebc996"
server_key_iv: "b77ce963e596aba8b53c8d0c01edfb63"
server_key_version: "encryptionkey@0"
I20260928 15:57:50.549875 30887 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 15:57:50.553083 30902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:50.554271 30887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 15:57:50.554456 30887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "114d760b6de344ae8c66ae12ce58ec91"
format_stamp: "Formatted at 2026-09-28 15:57:50 on dist-test-slave-9cqd"
server_key: "d0da9d9d7aa56220907fca8722ebc996"
server_key_iv: "b77ce963e596aba8b53c8d0c01edfb63"
server_key_version: "encryptionkey@0"
I20260928 15:57:50.554658 30887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:50.585707 30887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:50.590008 30887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:50.590224 30887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:50.592779 30887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:50.592862 30887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:50.593039 30887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:50.593096 30887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:57:50 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611070, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:50.617655 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.593305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:44471 (local address 127.22.125.254:44091)
0928 15:57:50.593484 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:50.593489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:50.595169 (+ 1680us) server_negotiation.cc:407] Connection header received
0928 15:57:50.597430 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:50.597440 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:50.597553 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:50.597672 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:50.599837 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.601195 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.603601 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.604280 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.604419 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.609572 (+ 5153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:50.609582 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:50.609590 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:50.609639 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:50.613148 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.614319 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.614325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.614330 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.614465 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.615135 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.615145 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.615147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.615566 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:50.615709 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:50.616832 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0928 15:57:50.617133 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 15:57:50.618237 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.594131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44091 (local address 127.22.125.194:44471)
0928 15:57:50.594988 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:50.595034 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:50.597008 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:50.597975 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:50.597993 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:50.598783 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:50.599592 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.599651 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.601398 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.601406 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.603432 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.603444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.604530 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.604539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.604787 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.605666 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:50.605687 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:50.609287 (+ 3600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:50.613386 (+ 4099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.613395 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.613413 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.614181 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.614596 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.614603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.614607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.614959 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.615798 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:50.615816 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:50.616353 (+ 537us) client_negotiation.cc:769] Sending connection context
0928 15:57:50.616704 (+ 351us) client_negotiation.cc:240] Negotiation successful
0928 15:57:50.617114 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":378,"threads_started":1}
I20260928 15:57:50.639590 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.624016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44091 (local address 127.22.125.194:49685)
0928 15:57:50.624766 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:50.624791 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:50.624947 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:50.625265 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:50.625272 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:50.625962 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:50.626445 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.626461 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.627407 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.627413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.629637 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:50.629659 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.630495 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.630503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:50.630738 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.631476 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:50.631493 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:50.633084 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:50.636689 (+ 3605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.636694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.636697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.637279 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.637778 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:50.637782 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:50.637785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:50.637967 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:50.638587 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:50.638592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:50.638813 (+ 221us) client_negotiation.cc:769] Sending connection context
0928 15:57:50.638967 (+ 154us) client_negotiation.cc:240] Negotiation successful
0928 15:57:50.639251 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":345,"threads_started":1}
I20260928 15:57:50.639925 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:50.624002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:49685 (local address 127.22.125.254:44091)
0928 15:57:50.624216 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:50.624224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:50.624887 (+ 663us) server_negotiation.cc:407] Connection header received
0928 15:57:50.625066 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:50.625069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:50.625126 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:50.625200 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:50.626598 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.627267 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.629800 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:50.630216 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:50.630384 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:50.633336 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:50.633344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:50.633348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:50.633386 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:50.636552 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.637461 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.637464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.637466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.637639 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:50.638110 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:50.638115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:50.638118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:50.638447 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:50.638609 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:50.639162 (+ 553us) server_negotiation.cc:299] Negotiation successful
0928 15:57:50.639470 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 15:57:50.648164 30887 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:38697
I20260928 15:57:50.648207 31021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:38697 every 8 connection(s)
I20260928 15:57:50.649400 30887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:50.651321 31022 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44091
I20260928 15:57:50.651561 31022 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:50.652252 31022 heartbeater.cc:511] Master 127.22.125.254:44091 requested a full tablet report, sending...
I20260928 15:57:50.653484 30676 ts_manager.cc:194] Registered new tserver with Master: 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697)
I20260928 15:57:50.654247 30676 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:44471
I20260928 15:57:50.656499 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 30887
I20260928 15:57:50.656657 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:50.657045 23031 external_mini_cluster.cc:1428] Setting key faf0b7b7508f480aba55e0ad08c1e3bc
WARNING: no policy specified for oryx/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab.
Entry for principal oryx/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab.
I20260928 15:57:50.702513 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44091
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab
--principal=oryx/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:37595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:51.019470 31026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:51.019786 31026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:51.019963 31026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:51.027272 31026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:51.027426 31026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:51.027577 31026 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 15:57:51.027719 31026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:51.037694 31026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab
--principal=oryx/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.125.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:51.039018 31026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:51.040405 31026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:51.053608 31031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:51.053656 31032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:51.053695 31034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:51.054482 31026 server_base.cc:1034] running on GCE node
I20260928 15:57:51.055063 31026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:51.055976 31026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:51.057251 31026 hybrid_clock.cc:648] HybridClock initialized: now 1790611071057176 us; error 56 us; skew 500 ppm
I20260928 15:57:51.057564 31026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:51 dist-test-slave-9cqd krb5kdc[30631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611071, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:51.060838 31026 init.cc:377] Logged in from keytab as oryx/127.22.125.195@KRBTEST.COM (short username oryx)
I20260928 15:57:51.062274 31026 webserver.cc:460] Webserver started at http://127.22.125.195:38411/ using document root <none> and password file <none>
I20260928 15:57:51.062947 31026 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:51.063061 31026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:51.063395 31026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:51.066499 31026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "c70bd93245e24bc999fc32991cb03a25"
format_stamp: "Formatted at 2026-09-28 15:57:51 on dist-test-slave-9cqd"
server_key: "8c17a76ac09870b45a2e6e2c5c3199f2"
server_key_iv: "9a71f81bc4337709f417ef8727bfa121"
server_key_version: "encryptionkey@0"
I20260928 15:57:51.067307 31026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "c70bd93245e24bc999fc32991cb03a25"
format_stamp: "Formatted at 2026-09-28 15:57:51 on dist-test-slave-9cqd"
server_key: "8c17a76ac09870b45a2e6e2c5c3199f2"
server_key_iv: "9a71f81bc4337709f417ef8727bfa121"
server_key_version: "encryptionkey@0"
I20260928 15:57:51.072383 31026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 15:57:51.075697 31041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:51.076992 31026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:51.077154 31026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "c70bd93245e24bc999fc32991cb03a25"
format_stamp: "Formatted at 2026-09-28 15:57:51 on dist-test-slave-9cqd"
server_key: "8c17a76ac09870b45a2e6e2c5c3199f2"
server_key_iv: "9a71f81bc4337709f417ef8727bfa121"
server_key_version: "encryptionkey@0"
I20260928 15:57:51.077359 31026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:51.107991 31026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:51.112115 31026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:51.112385 31026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:51.114929 31026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:51.115029 31026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:51.115154 31026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:51.115227 31026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 15:57:51 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611071, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:51.136292 30883 heartbeater.cc:503] Master 127.22.125.254:44091 was elected leader, sending a full tablet report...
I20260928 15:57:51.136729 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.115604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:43493 (local address 127.22.125.254:44091)
0928 15:57:51.115804 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.115813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.117279 (+ 1466us) server_negotiation.cc:407] Connection header received
0928 15:57:51.119450 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.119458 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.119541 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.119688 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:51.122242 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.123330 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.125664 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.126070 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.126207 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.129581 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:51.129587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:51.129590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:51.129648 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:51.132362 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.133520 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.133528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.133535 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.133703 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.134323 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.134332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.134348 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.134791 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:51.135027 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.135854 (+ 827us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.136199 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 15:57:51.137560 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.116353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44091 (local address 127.22.125.195:43493)
0928 15:57:51.117125 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.117162 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.119149 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.119999 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.120016 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.120978 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:51.122026 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.122059 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.124184 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.124191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.125520 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.125532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.126306 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.126314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.126524 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.127142 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:51.127163 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:51.129375 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:51.132497 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:51.132511 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:51.132521 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:51.133361 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:51.133869 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:51.133876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:51.133882 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:51.134176 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:51.134928 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:51.134942 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:51.135377 (+ 435us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.135690 (+ 313us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.136244 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":346,"threads_started":1}
I20260928 15:57:51.159567 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.143612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44091 (local address 127.22.125.195:50149)
0928 15:57:51.144457 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.144478 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.144623 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.145017 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.145025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.145733 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:51.146202 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.146218 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.147155 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.147163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.149293 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.149311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.150466 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.150471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.150629 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.151405 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:51.151436 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:51.152688 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:51.156727 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:51.156735 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:51.156746 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:51.157254 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:51.157697 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:51.157702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:51.157709 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:51.157907 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:51.158492 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:51.158497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:51.158699 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.158836 (+ 137us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.159259 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":416,"threads_started":1}
I20260928 15:57:51.159787 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.143473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:50149 (local address 127.22.125.254:44091)
0928 15:57:51.143624 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.143629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.144565 (+ 936us) server_negotiation.cc:407] Connection header received
0928 15:57:51.144759 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.144764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.144854 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.144960 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:51.146363 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.147017 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.149530 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.150081 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.150225 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.152923 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:51.152928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:51.152931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:51.152959 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:51.156564 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.157432 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.157438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.157441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.157581 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.158013 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.158018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.158024 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.158376 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:51.158534 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.159312 (+ 778us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.159466 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 15:57:51.173612 31026 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:34829
I20260928 15:57:51.173643 31160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:34829 every 8 connection(s)
I20260928 15:57:51.174749 31026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:51.176678 31161 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44091
I20260928 15:57:51.176911 31161 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:51.177510 31161 heartbeater.cc:511] Master 127.22.125.254:44091 requested a full tablet report, sending...
I20260928 15:57:51.178766 30676 ts_manager.cc:194] Registered new tserver with Master: c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:51.179528 30676 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:43493
I20260928 15:57:51.182241 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 31026
I20260928 15:57:51.182387 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:51.182729 23031 external_mini_cluster.cc:1428] Setting key a63d8d40eab25a9e70044406761bb3d8
I20260928 15:57:51.185439 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 15:57:51 dist-test-slave-9cqd krb5kdc[30631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 15:57:51 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:57:51 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 15:57:51.232157 30770 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:51.222167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.22.125.254:44091)
0928 15:57:51.222337 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.222342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.222620 (+ 278us) server_negotiation.cc:407] Connection header received
0928 15:57:51.222770 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.222774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.222839 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.222940 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:51.224235 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.224959 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.226199 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.226603 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.226692 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.231807 (+ 5115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40880: BlockingRecv error: recv got EOF from 127.0.0.1:40880 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 28 15:57:51 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:51.247404 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.235084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.22.125.254:44091)
0928 15:57:51.235288 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.235297 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.235568 (+ 271us) server_negotiation.cc:407] Connection header received
0928 15:57:51.235746 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.235751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.235834 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.235940 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:51.237010 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.237685 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.238900 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.239322 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.239455 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.242942 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:51.242948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:51.242951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:51.242979 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:51.245320 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.245889 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.245892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.245895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.246011 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.246318 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.246321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.246323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.246578 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:51.246774 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.246998 (+ 224us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.247155 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 15:57:51.251230 30676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:51.258044 30676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:51.280800 31184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.271845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53192 (local address 127.22.125.194:38697)
0928 15:57:51.272529 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.272537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.272560 (+ 23us) server_negotiation.cc:407] Connection header received
0928 15:57:51.272662 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.272671 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.272904 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.273111 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.273820 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.275347 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.277962 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.279273 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.279466 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.279630 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.280288 (+ 658us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.280507 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":453,"threads_started":1}
I20260928 15:57:51.280877 31179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.269467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39099 (local address 127.0.0.1:35588)
0928 15:57:51.270324 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.270392 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.270613 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.274277 (+ 3664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.274282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.274307 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.274660 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.274671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.276371 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.276379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.278024 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.278039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.279815 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.279822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.280088 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.280122 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.280233 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.280427 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":37,"thread_start_us":326,"threads_started":1}
I20260928 15:57:51.280920 31183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.271378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:38697 (local address 127.0.0.1:53192)
0928 15:57:51.271882 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.271900 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.272011 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.273128 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.273136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.273171 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.273638 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.273653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.275519 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.275530 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.277818 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.277827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.279594 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.279598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.279918 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.280075 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.280204 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.280412 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":125,"thread_start_us":670,"threads_started":1}
I20260928 15:57:51.281487 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.269821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.22.125.193:39099)
0928 15:57:51.273745 (+ 3924us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.273756 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.273781 (+ 25us) server_negotiation.cc:407] Connection header received
0928 15:57:51.273876 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.273883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.274089 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.274261 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.274820 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.276243 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.278181 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.279600 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.279773 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.279949 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.280591 (+ 642us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.280744 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3766,"thread_start_us":3618,"threads_started":1}
I20260928 15:57:51.281605 31181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.270179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:34829 (local address 127.0.0.1:53280)
0928 15:57:51.270791 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.270808 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.270937 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.272582 (+ 1645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.272591 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.272623 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.273031 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.273048 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.274945 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.274950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.277561 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.277575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.279563 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.279576 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.279918 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.280075 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.280206 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.280459 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":226,"thread_start_us":848,"threads_started":1}
I20260928 15:57:51.283118 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.270848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.22.125.195:34829)
0928 15:57:51.271804 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.271817 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.271856 (+ 39us) server_negotiation.cc:407] Connection header received
0928 15:57:51.271972 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.271982 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.272355 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.272548 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.273265 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.274810 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.277732 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.279039 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.279224 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.279425 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.282611 (+ 3186us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.282779 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":441,"threads_started":1}
I20260928 15:57:51.284583 30952 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac63cad48a405faa36419b2d6c7c3f (DEFAULT_TABLE table=test-table [id=43ef60a731df4a028a29f226f508b2d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:51.284838 30813 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac63cad48a405faa36419b2d6c7c3f (DEFAULT_TABLE table=test-table [id=43ef60a731df4a028a29f226f508b2d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:51.285880 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac63cad48a405faa36419b2d6c7c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.285995 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac63cad48a405faa36419b2d6c7c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.286489 31096 tablet_service.cc:1467] Processing CreateTablet for tablet 7eac63cad48a405faa36419b2d6c7c3f (DEFAULT_TABLE table=test-table [id=43ef60a731df4a028a29f226f508b2d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:51.287765 31096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eac63cad48a405faa36419b2d6c7c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.296597 31185 tablet_bootstrap.cc:492] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Bootstrap starting.
I20260928 15:57:51.299647 31185 tablet_bootstrap.cc:654] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.299922 31186 tablet_bootstrap.cc:492] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Bootstrap starting.
I20260928 15:57:51.301719 31185 log.cc:824] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:51.303267 31186 tablet_bootstrap.cc:654] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.304157 31187 tablet_bootstrap.cc:492] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Bootstrap starting.
I20260928 15:57:51.304456 31186 log.cc:824] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:51.304864 31185 tablet_bootstrap.cc:492] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: No bootstrap required, opened a new log
I20260928 15:57:51.305162 31185 ts_tablet_manager.cc:1397] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260928 15:57:51.306953 31187 tablet_bootstrap.cc:654] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.307332 31186 tablet_bootstrap.cc:492] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: No bootstrap required, opened a new log
I20260928 15:57:51.307597 31186 ts_tablet_manager.cc:1397] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260928 15:57:51.307972 31187 log.cc:824] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:51.310719 31187 tablet_bootstrap.cc:492] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: No bootstrap required, opened a new log
I20260928 15:57:51.310956 31187 ts_tablet_manager.cc:1397] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260928 15:57:51.315683 31185 raft_consensus.cc:348] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.316123 31185 raft_consensus.cc:374] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.316227 31185 raft_consensus.cc:729] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c14e55f7ad04080b2531520cb4225c5, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.316817 31185 consensus_queue.cc:260] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.317690 31186 raft_consensus.cc:348] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.318075 31186 raft_consensus.cc:374] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.318174 31186 raft_consensus.cc:729] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114d760b6de344ae8c66ae12ce58ec91, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.318544 31185 ts_tablet_manager.cc:1428] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260928 15:57:51.318821 31186 consensus_queue.cc:260] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.319077 31187 raft_consensus.cc:348] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.319502 31187 raft_consensus.cc:374] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.319597 31187 raft_consensus.cc:729] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70bd93245e24bc999fc32991cb03a25, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.319948 31022 heartbeater.cc:503] Master 127.22.125.254:44091 was elected leader, sending a full tablet report...
I20260928 15:57:51.320250 31187 consensus_queue.cc:260] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.320571 31186 ts_tablet_manager.cc:1428] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260928 15:57:51.322600 31161 heartbeater.cc:503] Master 127.22.125.254:44091 was elected leader, sending a full tablet report...
I20260928 15:57:51.323159 31187 ts_tablet_manager.cc:1428] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260928 15:57:51.380577 30884 tablet.cc:2369] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:57:51.401441 31023 tablet.cc:2369] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:57:51.426355 31162 tablet.cc:2369] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:57:51.472546 31191 raft_consensus.cc:482] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:51.472924 31191 raft_consensus.cc:504] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.473599 31192 raft_consensus.cc:482] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:51.473884 31192 raft_consensus.cc:504] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.474812 31191 leader_election.cc:288] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829), 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697)
I20260928 15:57:51.475476 31192 leader_election.cc:288] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099), c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:51.486874 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.475274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:37729 (local address 127.22.125.195:34829)
0928 15:57:51.475487 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.475494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.476101 (+ 607us) server_negotiation.cc:407] Connection header received
0928 15:57:51.476417 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.476426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.476531 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.476607 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.478055 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.479487 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.483107 (+ 3620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.484847 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.485032 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.485149 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.486181 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.486370 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 15:57:51.487779 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.475177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:34829 (local address 127.22.125.193:37729)
0928 15:57:51.475985 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.476006 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.476222 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.477252 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.477260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.477305 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.477848 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.477864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.479723 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.479731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.482884 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.482905 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.485512 (+ 2607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.485521 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.485889 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.485950 (+ 61us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.487273 (+ 1323us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.487413 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":528,"threads_started":1}
I20260928 15:57:51.489166 31116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac63cad48a405faa36419b2d6c7c3f" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25" is_pre_election: true
I20260928 15:57:51.489815 31116 raft_consensus.cc:2454] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c14e55f7ad04080b2531520cb4225c5 in term 0.
I20260928 15:57:51.490659 30766 leader_election.cc:302] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c14e55f7ad04080b2531520cb4225c5, c70bd93245e24bc999fc32991cb03a25; no voters:
I20260928 15:57:51.491101 31191 raft_consensus.cc:2781] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:51.491289 31191 raft_consensus.cc:482] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:51.491405 31191 raft_consensus.cc:3037] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:51.494931 31191 raft_consensus.cc:504] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.495141 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.487514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:35809 (local address 127.22.125.193:39099)
0928 15:57:51.487705 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.487711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.487729 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:57:51.487793 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.487801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.487903 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.487996 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.488741 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.489835 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.492528 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.493819 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.493940 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.494009 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.494746 (+ 737us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.494895 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 15:57:51.495559 31184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.476465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:57899 (local address 127.22.125.194:38697)
0928 15:57:51.477086 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.477095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.487279 (+ 10184us) server_negotiation.cc:407] Connection header received
0928 15:57:51.488156 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.488161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.488238 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.488303 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.489111 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.490285 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.493052 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.494304 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.494439 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.494549 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.495129 (+ 580us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.495278 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":308,"spinlock_wait_cycles":33024}
I20260928 15:57:51.495929 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.475916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:35159 (local address 127.22.125.195:34829)
0928 15:57:51.487154 (+ 11238us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.487161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.487761 (+ 600us) server_negotiation.cc:407] Connection header received
0928 15:57:51.488328 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.488333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.488406 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.488475 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.489378 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.490511 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.493370 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.494577 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.494694 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.494779 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.495500 (+ 721us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.495650 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11132,"thread_start_us":11841,"threads_started":1}
I20260928 15:57:51.496171 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.475890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39099 (local address 127.22.125.194:35809)
0928 15:57:51.486467 (+ 10577us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.486491 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.486676 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.488018 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.488023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.488082 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.488565 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.488577 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.490023 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.490027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.492340 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.492354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.494239 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.494246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.494426 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.494583 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.495357 (+ 774us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.495493 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10410,"mutex_wait_us":97,"thread_start_us":11441,"threads_started":1}
I20260928 15:57:51.496102 31116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac63cad48a405faa36419b2d6c7c3f" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25"
I20260928 15:57:51.495652 31191 leader_election.cc:288] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 election: Requested vote from peers c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829), 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697)
I20260928 15:57:51.496310 31116 raft_consensus.cc:3037] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:51.496732 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.476130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:38697 (local address 127.22.125.193:57899)
0928 15:57:51.487176 (+ 11046us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.487195 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.487967 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.488514 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.488520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.488544 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.488914 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.488930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.490460 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.490465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.492858 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.492875 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.494752 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.494758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.494954 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.494986 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.495727 (+ 741us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.495859 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10930,"spinlock_wait_cycles":34048,"thread_start_us":11167,"threads_started":1}
I20260928 15:57:51.497282 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.476129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:34829 (local address 127.22.125.194:35159)
0928 15:57:51.487653 (+ 11524us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.487674 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.488173 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.488681 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.488685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.488721 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.489108 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.489118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.490823 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.490888 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.493215 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.493227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.494954 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.494960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.495325 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.495358 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.496624 (+ 1266us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.496929 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11371,"thread_start_us":9887,"threads_started":1}
I20260928 15:57:51.497092 30833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac63cad48a405faa36419b2d6c7c3f" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c14e55f7ad04080b2531520cb4225c5" is_pre_election: true
I20260928 15:57:51.497579 30833 raft_consensus.cc:2379] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 114d760b6de344ae8c66ae12ce58ec91 in current term 1: Already voted for candidate 7c14e55f7ad04080b2531520cb4225c5 in this term.
I20260928 15:57:51.498001 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac63cad48a405faa36419b2d6c7c3f" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114d760b6de344ae8c66ae12ce58ec91"
I20260928 15:57:51.497942 30972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac63cad48a405faa36419b2d6c7c3f" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114d760b6de344ae8c66ae12ce58ec91" is_pre_election: true
I20260928 15:57:51.498411 30972 raft_consensus.cc:2454] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c14e55f7ad04080b2531520cb4225c5 in term 0.
I20260928 15:57:51.499253 31115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eac63cad48a405faa36419b2d6c7c3f" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25" is_pre_election: true
I20260928 15:57:51.499570 31116 raft_consensus.cc:2454] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c14e55f7ad04080b2531520cb4225c5 in term 1.
I20260928 15:57:51.499945 30766 leader_election.cc:302] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c14e55f7ad04080b2531520cb4225c5, c70bd93245e24bc999fc32991cb03a25; no voters: 114d760b6de344ae8c66ae12ce58ec91
I20260928 15:57:51.500104 30906 leader_election.cc:302] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 114d760b6de344ae8c66ae12ce58ec91; no voters: 7c14e55f7ad04080b2531520cb4225c5, c70bd93245e24bc999fc32991cb03a25
I20260928 15:57:51.500252 31191 raft_consensus.cc:2781] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:51.500494 31192 raft_consensus.cc:3037] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:51.502540 31192 raft_consensus.cc:2726] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260928 15:57:51.502940 31191 raft_consensus.cc:686] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 1 LEADER]: Becoming Leader. State: Replica: 7c14e55f7ad04080b2531520cb4225c5, State: Running, Role: LEADER
I20260928 15:57:51.503453 31191 consensus_queue.cc:237] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } }
I20260928 15:57:51.507881 30674 catalog_manager.cc:5495] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193). New cstate: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:51.578171 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.572904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.22.125.193:39099)
0928 15:57:51.573172 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.573176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.573195 (+ 19us) server_negotiation.cc:407] Connection header received
0928 15:57:51.573381 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.573386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.573466 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.573596 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:51.573902 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.574627 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.575845 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.576489 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.576587 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.577010 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:51.577569 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:51.577684 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.577755 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.577824 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 15:57:51.586328 30971 raft_consensus.cc:1261] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Refusing update from remote peer 7c14e55f7ad04080b2531520cb4225c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:51.586328 31115 raft_consensus.cc:1261] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Refusing update from remote peer 7c14e55f7ad04080b2531520cb4225c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:51.587177 31199 consensus_queue.cc:1035] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:51.587540 31191 consensus_queue.cc:1035] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:51.599025 31202 mvcc.cc:205] Tried to move back new op lower bound from 7334342949208571904 to 7334342948881879040. Current Snapshot: MvccSnapshot[applied={T|T < 7334342949208571904}]
I20260928 15:57:51.611299 30676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:51.611616 30676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:51.615166 30676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Sending DeleteTablet for 3 replicas of tablet 7eac63cad48a405faa36419b2d6c7c3f
I20260928 15:57:51.616329 30813 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eac63cad48a405faa36419b2d6c7c3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:51 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:35588
I20260928 15:57:51.616451 30952 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eac63cad48a405faa36419b2d6c7c3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:51 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53192
I20260928 15:57:51.617080 31096 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eac63cad48a405faa36419b2d6c7c3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:51 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53280
I20260928 15:57:51.617290 31212 tablet_replica.cc:330] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: stopping tablet replica
I20260928 15:57:51.617691 31212 raft_consensus.cc:2229] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:51.618320 31212 raft_consensus.cc:2258] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:51.618067 30674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:51.618902 30674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:51.619035 31213 tablet_replica.cc:330] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: stopping tablet replica
I20260928 15:57:51.619408 31213 raft_consensus.cc:2229] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:51.619912 31213 raft_consensus.cc:2258] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:51.621094 31214 tablet_replica.cc:330] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: stopping tablet replica
I20260928 15:57:51.621479 31214 raft_consensus.cc:2229] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:51.621941 31214 raft_consensus.cc:2258] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:51.622228 31213 ts_tablet_manager.cc:1905] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:51.622449 31212 ts_tablet_manager.cc:1905] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:51.627615 30952 tablet_service.cc:1467] Processing CreateTablet for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e (DEFAULT_TABLE table=test-table [id=e4291d85608b4a3d9023f436781d7030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:51.628098 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.629330 31213 ts_tablet_manager.cc:1918] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:51.629508 31213 log.cc:1196] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/7eac63cad48a405faa36419b2d6c7c3f
I20260928 15:57:51.630111 31213 ts_tablet_manager.cc:1939] T 7eac63cad48a405faa36419b2d6c7c3f P 114d760b6de344ae8c66ae12ce58ec91: Deleting consensus metadata
I20260928 15:57:51.630749 31212 ts_tablet_manager.cc:1918] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:51.630975 31212 log.cc:1196] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/7eac63cad48a405faa36419b2d6c7c3f
I20260928 15:57:51.631556 31212 ts_tablet_manager.cc:1939] T 7eac63cad48a405faa36419b2d6c7c3f P 7c14e55f7ad04080b2531520cb4225c5: Deleting consensus metadata
I20260928 15:57:51.632081 30663 catalog_manager.cc:4841] TS 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697): tablet 7eac63cad48a405faa36419b2d6c7c3f (table test-table [id=43ef60a731df4a028a29f226f508b2d9]) successfully deleted
I20260928 15:57:51.632797 30813 tablet_service.cc:1467] Processing CreateTablet for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e (DEFAULT_TABLE table=test-table [id=e4291d85608b4a3d9023f436781d7030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:51.632809 31096 tablet_service.cc:1467] Processing CreateTablet for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e (DEFAULT_TABLE table=test-table [id=e4291d85608b4a3d9023f436781d7030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:51.633244 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.633262 31096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.634770 30660 catalog_manager.cc:4841] TS 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099): tablet 7eac63cad48a405faa36419b2d6c7c3f (table test-table [id=43ef60a731df4a028a29f226f508b2d9]) successfully deleted
I20260928 15:57:51.636461 31186 tablet_bootstrap.cc:492] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: Bootstrap starting.
I20260928 15:57:51.638120 31186 tablet_bootstrap.cc:654] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.641259 31186 tablet_bootstrap.cc:492] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: No bootstrap required, opened a new log
I20260928 15:57:51.641434 31186 ts_tablet_manager.cc:1397] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 15:57:51.641894 31185 tablet_bootstrap.cc:492] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: Bootstrap starting.
I20260928 15:57:51.641882 31214 ts_tablet_manager.cc:1905] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:51.642367 31186 raft_consensus.cc:348] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.642652 31186 raft_consensus.cc:374] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.642792 31186 raft_consensus.cc:729] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114d760b6de344ae8c66ae12ce58ec91, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.643134 31186 consensus_queue.cc:260] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.643620 31185 tablet_bootstrap.cc:654] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.643821 31186 ts_tablet_manager.cc:1428] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:51.648979 31214 ts_tablet_manager.cc:1918] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:51.650841 31214 log.cc:1196] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/7eac63cad48a405faa36419b2d6c7c3f
I20260928 15:57:51.650954 31187 tablet_bootstrap.cc:492] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: Bootstrap starting.
I20260928 15:57:51.651342 31185 tablet_bootstrap.cc:492] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: No bootstrap required, opened a new log
I20260928 15:57:51.651490 31185 ts_tablet_manager.cc:1397] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260928 15:57:51.651600 31214 ts_tablet_manager.cc:1939] T 7eac63cad48a405faa36419b2d6c7c3f P c70bd93245e24bc999fc32991cb03a25: Deleting consensus metadata
I20260928 15:57:51.653559 31185 raft_consensus.cc:348] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.653806 31187 tablet_bootstrap.cc:654] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.653894 31185 raft_consensus.cc:374] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.654053 31185 raft_consensus.cc:729] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c14e55f7ad04080b2531520cb4225c5, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.654410 31185 consensus_queue.cc:260] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.655050 31185 ts_tablet_manager.cc:1428] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 15:57:51.659801 30663 catalog_manager.cc:4841] TS c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829): tablet 7eac63cad48a405faa36419b2d6c7c3f (table test-table [id=43ef60a731df4a028a29f226f508b2d9]) successfully deleted
I20260928 15:57:51.661010 31187 tablet_bootstrap.cc:492] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: No bootstrap required, opened a new log
I20260928 15:57:51.661183 31187 ts_tablet_manager.cc:1397] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.005s
I20260928 15:57:51.661909 31187 raft_consensus.cc:348] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.662142 31187 raft_consensus.cc:374] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.662241 31187 raft_consensus.cc:729] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70bd93245e24bc999fc32991cb03a25, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.662485 31187 consensus_queue.cc:260] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.663045 31187 ts_tablet_manager.cc:1428] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 15:57:51.777423 31193 raft_consensus.cc:482] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:51.777673 31193 raft_consensus.cc:504] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.779196 31193 leader_election.cc:288] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099), 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697)
I20260928 15:57:51.786846 31184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.779851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:41755 (local address 127.22.125.194:38697)
0928 15:57:51.780028 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.780034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.780546 (+ 512us) server_negotiation.cc:407] Connection header received
0928 15:57:51.780739 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.780745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.780819 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.780916 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.781685 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.782807 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.785164 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.785909 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.786072 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.786176 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.786445 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.786576 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 15:57:51.786948 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.779491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:38697 (local address 127.22.125.195:41755)
0928 15:57:51.780429 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.780459 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.780592 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.780956 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.780962 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.780994 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.781481 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.781494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.782945 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.782950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.785033 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.785042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.786076 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.786082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.786285 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.786309 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.786418 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.786541 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":18,"thread_start_us":647,"threads_started":1}
I20260928 15:57:51.787571 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f9e74a6c264cbe8aaa2fc8e99f2e6e" candidate_uuid: "c70bd93245e24bc999fc32991cb03a25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114d760b6de344ae8c66ae12ce58ec91" is_pre_election: true
I20260928 15:57:51.787847 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.779604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:50727 (local address 127.22.125.193:39099)
0928 15:57:51.779806 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.779814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.780121 (+ 307us) server_negotiation.cc:407] Connection header received
0928 15:57:51.780504 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.780509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.780597 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.780699 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:57:51.781296 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.782432 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.784998 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.786308 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.786466 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.786534 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.787365 (+ 831us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.787546 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 15:57:51.787844 30971 raft_consensus.cc:2454] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70bd93245e24bc999fc32991cb03a25 in term 0.
I20260928 15:57:51.788156 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.779396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39099 (local address 127.22.125.195:50727)
0928 15:57:51.779973 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.779996 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.780258 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.780767 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.780771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.780795 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:57:51.781123 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.781145 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.782576 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.782581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.784811 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.784830 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.786692 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.786699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.786880 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.787203 (+ 323us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.787311 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.787502 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":96,"thread_start_us":437,"threads_started":1}
I20260928 15:57:51.788343 31045 leader_election.cc:302] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114d760b6de344ae8c66ae12ce58ec91, c70bd93245e24bc999fc32991cb03a25; no voters:
I20260928 15:57:51.788735 31193 raft_consensus.cc:2781] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:51.788898 31193 raft_consensus.cc:482] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:51.788839 30833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f9e74a6c264cbe8aaa2fc8e99f2e6e" candidate_uuid: "c70bd93245e24bc999fc32991cb03a25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c14e55f7ad04080b2531520cb4225c5" is_pre_election: true
I20260928 15:57:51.788991 31193 raft_consensus.cc:3037] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:51.789080 30833 raft_consensus.cc:2454] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70bd93245e24bc999fc32991cb03a25 in term 0.
I20260928 15:57:51.791204 31193 raft_consensus.cc:504] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.791785 31193 leader_election.cc:288] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [CANDIDATE]: Term 1 election: Requested vote from peers 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099), 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697)
I20260928 15:57:51.792223 30833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f9e74a6c264cbe8aaa2fc8e99f2e6e" candidate_uuid: "c70bd93245e24bc999fc32991cb03a25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c14e55f7ad04080b2531520cb4225c5"
I20260928 15:57:51.792289 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f9e74a6c264cbe8aaa2fc8e99f2e6e" candidate_uuid: "c70bd93245e24bc999fc32991cb03a25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114d760b6de344ae8c66ae12ce58ec91"
I20260928 15:57:51.792428 30833 raft_consensus.cc:3037] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:51.792516 30971 raft_consensus.cc:3037] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:51.794656 30833 raft_consensus.cc:2454] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70bd93245e24bc999fc32991cb03a25 in term 1.
I20260928 15:57:51.795058 31042 leader_election.cc:302] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c14e55f7ad04080b2531520cb4225c5, c70bd93245e24bc999fc32991cb03a25; no voters:
I20260928 15:57:51.795316 31193 raft_consensus.cc:2781] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:51.795466 30971 raft_consensus.cc:2454] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70bd93245e24bc999fc32991cb03a25 in term 1.
I20260928 15:57:51.795853 31193 raft_consensus.cc:686] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 1 LEADER]: Becoming Leader. State: Replica: c70bd93245e24bc999fc32991cb03a25, State: Running, Role: LEADER
I20260928 15:57:51.796474 31193 consensus_queue.cc:237] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.799993 30674 catalog_manager.cc:5495] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 reported cstate change: term changed from 0 to 1, leader changed from <none> to c70bd93245e24bc999fc32991cb03a25 (127.22.125.195). New cstate: current_term: 1 leader_uuid: "c70bd93245e24bc999fc32991cb03a25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } health_report { overall_health: HEALTHY } } }
Sep 28 15:57:51 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611069, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.254@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:51.891806 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.874421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.22.125.254:44091)
0928 15:57:51.876939 (+ 2518us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:51.876943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:51.876964 (+ 21us) server_negotiation.cc:407] Connection header received
0928 15:57:51.877009 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:51.877013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:51.877070 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:51.877159 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:51.878661 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.879557 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.881438 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.881982 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.882083 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.886091 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:51.886096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:51.886099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:51.886141 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:51.888804 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.889714 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.889719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.889721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.889835 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:51.890295 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:51.890299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:51.890300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:51.890562 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:51.890671 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:51.891190 (+ 519us) server_negotiation.cc:299] Negotiation successful
0928 15:57:51.891516 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":160,"server-negotiator.queue_time_us":2398,"thread_start_us":2265,"threads_started":1}
I20260928 15:57:51.891841 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:51.874384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44091 (local address 127.0.0.1:40890)
0928 15:57:51.876626 (+ 2242us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:51.876651 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:51.876809 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:51.877248 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:51.877255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:51.878154 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:51.878431 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.878446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.879701 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.879706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.881245 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:51.881262 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:51.882162 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:51.882169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:51.882337 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:51.883048 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:51.883071 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:51.885882 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:51.888947 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:51.888960 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:51.888967 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:51.889555 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:51.889974 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:51.889980 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:51.889983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:51.890173 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:51.890714 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:51.890723 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:51.890913 (+ 190us) client_negotiation.cc:769] Sending connection context
0928 15:57:51.891101 (+ 188us) client_negotiation.cc:240] Negotiation successful
0928 15:57:51.891401 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2021,"thread_start_us":1858,"threads_started":1}
I20260928 15:57:51.897114 30674 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:40890:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 15:57:51.898113 30674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:51.906399 30813 tablet_service.cc:1467] Processing CreateTablet for tablet 873db6e0c972458089125726f1dea52d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d93f069666c6488a87502970b0400376]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:57:51.907040 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 873db6e0c972458089125726f1dea52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.906843 31096 tablet_service.cc:1467] Processing CreateTablet for tablet 873db6e0c972458089125726f1dea52d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d93f069666c6488a87502970b0400376]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:57:51.907457 31096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 873db6e0c972458089125726f1dea52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.912844 30952 tablet_service.cc:1467] Processing CreateTablet for tablet 873db6e0c972458089125726f1dea52d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d93f069666c6488a87502970b0400376]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 15:57:51.913383 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 873db6e0c972458089125726f1dea52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:51.914887 31187 tablet_bootstrap.cc:492] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: Bootstrap starting.
I20260928 15:57:51.916584 31187 tablet_bootstrap.cc:654] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.919260 31185 tablet_bootstrap.cc:492] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: Bootstrap starting.
I20260928 15:57:51.921104 31185 tablet_bootstrap.cc:654] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.922209 31186 tablet_bootstrap.cc:492] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: Bootstrap starting.
I20260928 15:57:51.924782 31186 tablet_bootstrap.cc:654] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:51.926857 31187 tablet_bootstrap.cc:492] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: No bootstrap required, opened a new log
I20260928 15:57:51.926913 31185 tablet_bootstrap.cc:492] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: No bootstrap required, opened a new log
I20260928 15:57:51.927021 31187 ts_tablet_manager.cc:1397] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260928 15:57:51.927111 31185 ts_tablet_manager.cc:1397] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 15:57:51.928031 31187 raft_consensus.cc:348] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.928305 31187 raft_consensus.cc:374] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.928126 31185 raft_consensus.cc:348] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.928382 31187 raft_consensus.cc:729] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70bd93245e24bc999fc32991cb03a25, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.928462 31185 raft_consensus.cc:374] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.928555 31185 raft_consensus.cc:729] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c14e55f7ad04080b2531520cb4225c5, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.928648 31187 consensus_queue.cc:260] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.929231 31187 ts_tablet_manager.cc:1428] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 15:57:51.929457 31193 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:51.929733 31193 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:51.930361 31185 consensus_queue.cc:260] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.930795 31186 tablet_bootstrap.cc:492] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: No bootstrap required, opened a new log
I20260928 15:57:51.930964 31186 ts_tablet_manager.cc:1397] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20260928 15:57:51.930871 31207 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:51.931217 31207 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:51.931663 31186 raft_consensus.cc:348] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.931950 31186 raft_consensus.cc:374] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:51.932070 31186 raft_consensus.cc:729] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114d760b6de344ae8c66ae12ce58ec91, State: Initialized, Role: FOLLOWER
I20260928 15:57:51.932328 31185 ts_tablet_manager.cc:1428] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260928 15:57:51.932389 31186 consensus_queue.cc:260] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:51.932951 31192 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:51.933228 31192 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:51.933686 31186 ts_tablet_manager.cc:1428] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 15:57:52.110565 31207 raft_consensus.cc:482] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:52.110791 31207 raft_consensus.cc:504] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.111487 31207 leader_election.cc:288] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697), c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:52.112098 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "873db6e0c972458089125726f1dea52d" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114d760b6de344ae8c66ae12ce58ec91" is_pre_election: true
I20260928 15:57:52.112149 31115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "873db6e0c972458089125726f1dea52d" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25" is_pre_election: true
I20260928 15:57:52.112391 30971 raft_consensus.cc:2454] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c14e55f7ad04080b2531520cb4225c5 in term 0.
I20260928 15:57:52.112416 31115 raft_consensus.cc:2454] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c14e55f7ad04080b2531520cb4225c5 in term 0.
I20260928 15:57:52.112802 30766 leader_election.cc:302] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c14e55f7ad04080b2531520cb4225c5, c70bd93245e24bc999fc32991cb03a25; no voters:
I20260928 15:57:52.113059 31207 raft_consensus.cc:2781] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:52.113183 31207 raft_consensus.cc:482] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:52.113277 31207 raft_consensus.cc:3037] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.115195 31207 raft_consensus.cc:504] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.115708 31207 leader_election.cc:288] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 election: Requested vote from peers 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697), c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:52.116147 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "873db6e0c972458089125726f1dea52d" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114d760b6de344ae8c66ae12ce58ec91"
I20260928 15:57:52.116266 31115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "873db6e0c972458089125726f1dea52d" candidate_uuid: "7c14e55f7ad04080b2531520cb4225c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25"
I20260928 15:57:52.116348 30971 raft_consensus.cc:3037] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.116490 31115 raft_consensus.cc:3037] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.118237 30971 raft_consensus.cc:2454] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c14e55f7ad04080b2531520cb4225c5 in term 1.
I20260928 15:57:52.118454 31115 raft_consensus.cc:2454] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c14e55f7ad04080b2531520cb4225c5 in term 1.
I20260928 15:57:52.118626 30766 leader_election.cc:302] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114d760b6de344ae8c66ae12ce58ec91, 7c14e55f7ad04080b2531520cb4225c5; no voters:
I20260928 15:57:52.118916 31207 raft_consensus.cc:2781] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:52.119297 31207 raft_consensus.cc:686] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [term 1 LEADER]: Becoming Leader. State: Replica: 7c14e55f7ad04080b2531520cb4225c5, State: Running, Role: LEADER
I20260928 15:57:52.119580 31207 consensus_queue.cc:237] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.121361 31232 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: TxnStatusTablet state changed. Reason: New leader 7c14e55f7ad04080b2531520cb4225c5. Latest consensus state: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:52.121614 31232 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:57:52.122138 31234 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 15:57:52.122308 31234 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 15:57:52.122326 30676 catalog_manager.cc:5495] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193). New cstate: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:52.182080 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.174848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39099 (local address 127.0.0.1:35608)
0928 15:57:52.175408 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:52.175426 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:52.175615 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:52.176160 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:52.176166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:52.176179 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:52.176385 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.176397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.177852 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.177858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.179435 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.179450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.180561 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.180568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.180804 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.180887 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.181289 (+ 402us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.181309 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 15:57:52.181401 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 15:57:52.181752 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408}
I20260928 15:57:52.183421 31193 consensus_queue.cc:1035] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.184118 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.174847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.22.125.193:39099)
0928 15:57:52.175200 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.175208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.175534 (+ 326us) server_negotiation.cc:407] Connection header received
0928 15:57:52.175883 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.175889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.175979 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.176120 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:52.176549 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.177652 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.179609 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.180391 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.180525 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.181058 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.181182 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.181269 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.181442 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.183169 (+ 1727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260928 15:57:52.195256 31115 raft_consensus.cc:1261] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Refusing update from remote peer 7c14e55f7ad04080b2531520cb4225c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:52.195900 30971 raft_consensus.cc:1261] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Refusing update from remote peer 7c14e55f7ad04080b2531520cb4225c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:52.195813 31207 consensus_queue.cc:1035] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.197278 31232 consensus_queue.cc:1035] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.209543 31220 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: TxnStatusTablet state changed. Reason: New leader 7c14e55f7ad04080b2531520cb4225c5. Latest consensus state: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:52.209901 31220 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:52.210065 31220 consensus_queue.cc:1035] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.213037 31238 mvcc.cc:205] Tried to move back new op lower bound from 7334342951705968640 to 7334342951404535808. Current Snapshot: MvccSnapshot[applied={T|T < 7334342951705968640}]
I20260928 15:57:52.216753 31207 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:52.217075 31207 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:57:52.221566 31239 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:52.222045 31239 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P c70bd93245e24bc999fc32991cb03a25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:52.233780 31192 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: TxnStatusTablet state changed. Reason: New leader 7c14e55f7ad04080b2531520cb4225c5. Latest consensus state: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:52.234138 31192 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:52.235227 31232 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:52.235509 31232 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P 7c14e55f7ad04080b2531520cb4225c5: This TxnStatusTablet replica's current role is: LEADER
I20260928 15:57:52.238085 31241 mvcc.cc:205] Tried to move back new op lower bound from 7334342951705968640 to 7334342951404535808. Current Snapshot: MvccSnapshot[applied={T|T < 7334342951705968640}]
I20260928 15:57:52.238838 31192 tablet_replica.cc:439] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c14e55f7ad04080b2531520cb4225c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } } }
I20260928 15:57:52.239086 31192 tablet_replica.cc:442] T 873db6e0c972458089125726f1dea52d P 114d760b6de344ae8c66ae12ce58ec91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 15:57:52.244684 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.233582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.22.125.195:34829)
0928 15:57:52.234961 (+ 1379us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.234967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.238579 (+ 3612us) server_negotiation.cc:407] Connection header received
0928 15:57:52.238679 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.238684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.238797 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.238967 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:52.239933 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.240772 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.242194 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.242648 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.242781 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.243231 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.243907 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.244093 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.244182 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.244260 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261,"thread_start_us":1150,"threads_started":1}
I20260928 15:57:52.260792 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.254641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:39125 (local address 127.22.125.193:39099)
0928 15:57:52.254796 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.254800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.255353 (+ 553us) server_negotiation.cc:407] Connection header received
0928 15:57:52.255575 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.255581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.255656 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.255749 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:52.256305 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.257155 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.258858 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.259364 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.259529 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.260027 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.260181 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.260302 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.260421 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.260501 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 15:57:52.260993 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.254668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39099 (local address 127.22.125.195:39125)
0928 15:57:52.255250 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:52.255265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:52.255429 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:52.255772 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:52.255776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:52.255789 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:52.256152 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.256166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.257287 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.257293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.258708 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.258720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.259544 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.259550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.259732 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.259832 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.260301 (+ 469us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.260314 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 15:57:52.260397 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 15:57:52.260525 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":303,"threads_started":1}
I20260928 15:57:52.301458 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.292523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:36463 (local address 127.22.125.195:34829)
0928 15:57:52.292730 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.292765 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.293517 (+ 752us) server_negotiation.cc:407] Connection header received
0928 15:57:52.293937 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.293942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.294045 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.294128 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:52.295063 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.296330 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.298784 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.299659 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.299801 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.300643 (+ 842us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.300793 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.300867 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.301085 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.301168 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 15:57:52.302212 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.292360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:34829 (local address 127.22.125.193:36463)
0928 15:57:52.293413 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:52.293434 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:52.293602 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:52.294411 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:52.294416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:52.294429 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:52.294844 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.294856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.296493 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.296499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.298621 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.298637 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.300134 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.300140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.300341 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.300419 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.300950 (+ 531us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.300961 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 15:57:52.301626 (+ 665us) client_negotiation.cc:240] Negotiation successful
0928 15:57:52.301780 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":628,"threads_started":1}
I20260928 15:57:52.349727 30674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:52.350019 30674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:52.353554 30674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Sending DeleteTablet for 3 replicas of tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e
I20260928 15:57:52.354617 30952 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53192
I20260928 15:57:52.355232 31259 tablet_replica.cc:330] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: stopping tablet replica
I20260928 15:57:52.355265 31096 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53280
I20260928 15:57:52.355458 31259 raft_consensus.cc:2229] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:52.355695 31259 raft_consensus.cc:2258] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.355712 30813 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:35588
I20260928 15:57:52.356834 31259 ts_tablet_manager.cc:1905] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.367743 31259 ts_tablet_manager.cc:1918] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:52.367879 31259 log.cc:1196] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/d5f9e74a6c264cbe8aaa2fc8e99f2e6e
I20260928 15:57:52.368445 31259 ts_tablet_manager.cc:1939] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 114d760b6de344ae8c66ae12ce58ec91: Deleting consensus metadata
I20260928 15:57:52.369588 30663 catalog_manager.cc:4841] TS 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697): tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e (table test-table [id=e4291d85608b4a3d9023f436781d7030]) successfully deleted
I20260928 15:57:52.372887 31261 tablet_replica.cc:330] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: stopping tablet replica
I20260928 15:57:52.373221 31261 raft_consensus.cc:2229] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:52.373679 31261 raft_consensus.cc:2258] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.380616 31260 tablet_replica.cc:330] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: stopping tablet replica
I20260928 15:57:52.380826 31260 raft_consensus.cc:2229] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:52.381528 31260 raft_consensus.cc:2258] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.381605 31261 ts_tablet_manager.cc:1905] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.382984 31260 ts_tablet_manager.cc:1905] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.389827 31260 ts_tablet_manager.cc:1918] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:52.389971 31260 log.cc:1196] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/d5f9e74a6c264cbe8aaa2fc8e99f2e6e
I20260928 15:57:52.390453 31260 ts_tablet_manager.cc:1939] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P c70bd93245e24bc999fc32991cb03a25: Deleting consensus metadata
I20260928 15:57:52.391534 30663 catalog_manager.cc:4841] TS c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829): tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e (table test-table [id=e4291d85608b4a3d9023f436781d7030]) successfully deleted
I20260928 15:57:52.396334 31261 ts_tablet_manager.cc:1918] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:52.396481 31261 log.cc:1196] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/d5f9e74a6c264cbe8aaa2fc8e99f2e6e
I20260928 15:57:52.396986 31261 ts_tablet_manager.cc:1939] T d5f9e74a6c264cbe8aaa2fc8e99f2e6e P 7c14e55f7ad04080b2531520cb4225c5: Deleting consensus metadata
I20260928 15:57:52.398166 30660 catalog_manager.cc:4841] TS 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099): tablet d5f9e74a6c264cbe8aaa2fc8e99f2e6e (table test-table [id=e4291d85608b4a3d9023f436781d7030]) successfully deleted
Sep 28 15:57:52 dist-test-slave-9cqd krb5kdc[30631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 15:57:52 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:57:52 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 15:57:52.452697 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:57:52.423169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.22.125.254:44091)
0928 15:57:52.439925 (+ 16756us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.439932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.439954 (+ 22us) server_negotiation.cc:407] Connection header received
0928 15:57:52.440021 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.440027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.440184 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.440304 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:52.442184 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.443425 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.445389 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.446034 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.446207 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.452394 (+ 6187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57970: BlockingRecv error: recv got EOF from 127.0.0.1:57970 (error 108)
Metrics: {"server-negotiator.queue_time_us":16614,"thread_start_us":16518,"threads_started":1}
Sep 28 15:57:52 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM
I20260928 15:57:52.475467 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.456201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.22.125.254:44091)
0928 15:57:52.456384 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.456390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.458079 (+ 1689us) server_negotiation.cc:407] Connection header received
0928 15:57:52.458233 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.458238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.458317 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.458424 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:52.459955 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.461132 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.463000 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.463632 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.463756 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.468735 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:52.468743 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:52.468748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:52.468792 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:52.472583 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:52.473396 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:52.473402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:52.473406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:52.473552 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:52.473954 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:52.473960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:52.473963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:52.474296 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:52.474574 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.474892 (+ 318us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.475121 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 15:57:52.479974 30674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:52.480737 30674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:52.489078 30813 tablet_service.cc:1467] Processing CreateTablet for tablet e0476492ee3c404f883afe1d53c66ae6 (DEFAULT_TABLE table=test-table [id=a0790cfb902746cdb88af83a51267cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:52.489261 30952 tablet_service.cc:1467] Processing CreateTablet for tablet e0476492ee3c404f883afe1d53c66ae6 (DEFAULT_TABLE table=test-table [id=a0790cfb902746cdb88af83a51267cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:52.489629 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0476492ee3c404f883afe1d53c66ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:52.489677 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0476492ee3c404f883afe1d53c66ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:52.492692 31096 tablet_service.cc:1467] Processing CreateTablet for tablet e0476492ee3c404f883afe1d53c66ae6 (DEFAULT_TABLE table=test-table [id=a0790cfb902746cdb88af83a51267cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:52.493141 31096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0476492ee3c404f883afe1d53c66ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:52.504499 31275 tablet_bootstrap.cc:492] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: Bootstrap starting.
I20260928 15:57:52.505791 31275 tablet_bootstrap.cc:654] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:52.506546 31277 tablet_bootstrap.cc:492] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: Bootstrap starting.
I20260928 15:57:52.508471 31277 tablet_bootstrap.cc:654] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:52.509712 31275 tablet_bootstrap.cc:492] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: No bootstrap required, opened a new log
I20260928 15:57:52.509907 31275 ts_tablet_manager.cc:1397] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260928 15:57:52.510679 31275 raft_consensus.cc:348] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.511093 31275 raft_consensus.cc:374] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:52.511229 31275 raft_consensus.cc:729] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114d760b6de344ae8c66ae12ce58ec91, State: Initialized, Role: FOLLOWER
I20260928 15:57:52.511528 31275 consensus_queue.cc:260] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.513674 31276 tablet_bootstrap.cc:492] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: Bootstrap starting.
I20260928 15:57:52.515596 31275 ts_tablet_manager.cc:1428] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260928 15:57:52.515746 31276 tablet_bootstrap.cc:654] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:52.521147 31276 tablet_bootstrap.cc:492] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: No bootstrap required, opened a new log
I20260928 15:57:52.521319 31276 ts_tablet_manager.cc:1397] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260928 15:57:52.522274 31277 tablet_bootstrap.cc:492] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: No bootstrap required, opened a new log
I20260928 15:57:52.522450 31277 ts_tablet_manager.cc:1397] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260928 15:57:52.523276 31277 raft_consensus.cc:348] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.523535 31277 raft_consensus.cc:374] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:52.523622 31277 raft_consensus.cc:729] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70bd93245e24bc999fc32991cb03a25, State: Initialized, Role: FOLLOWER
I20260928 15:57:52.523885 31277 consensus_queue.cc:260] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.526805 31277 ts_tablet_manager.cc:1428] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 15:57:52.530260 31276 raft_consensus.cc:348] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.530539 31276 raft_consensus.cc:374] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:52.530653 31276 raft_consensus.cc:729] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c14e55f7ad04080b2531520cb4225c5, State: Initialized, Role: FOLLOWER
I20260928 15:57:52.530978 31276 consensus_queue.cc:260] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.533833 31276 ts_tablet_manager.cc:1428] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260928 15:57:52.681315 31192 raft_consensus.cc:482] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:52.681553 31192 raft_consensus.cc:504] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.682225 31192 leader_election.cc:288] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099), c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:52.682899 31115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0476492ee3c404f883afe1d53c66ae6" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25" is_pre_election: true
I20260928 15:57:52.682880 30833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0476492ee3c404f883afe1d53c66ae6" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c14e55f7ad04080b2531520cb4225c5" is_pre_election: true
I20260928 15:57:52.683144 31115 raft_consensus.cc:2454] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 0.
I20260928 15:57:52.683172 30833 raft_consensus.cc:2454] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 0.
I20260928 15:57:52.683663 30906 leader_election.cc:302] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114d760b6de344ae8c66ae12ce58ec91, c70bd93245e24bc999fc32991cb03a25; no voters:
I20260928 15:57:52.683964 31192 raft_consensus.cc:2781] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:52.684113 31192 raft_consensus.cc:482] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:52.684239 31192 raft_consensus.cc:3037] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.687789 31192 raft_consensus.cc:504] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.688994 31115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0476492ee3c404f883afe1d53c66ae6" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25"
I20260928 15:57:52.689251 31115 raft_consensus.cc:3037] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.689838 30833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0476492ee3c404f883afe1d53c66ae6" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c14e55f7ad04080b2531520cb4225c5"
I20260928 15:57:52.690100 30833 raft_consensus.cc:3037] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.692296 30833 raft_consensus.cc:2454] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 1.
I20260928 15:57:52.692625 31192 leader_election.cc:288] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 election: Requested vote from peers 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099), c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:52.692623 31115 raft_consensus.cc:2454] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 1.
I20260928 15:57:52.692741 30903 leader_election.cc:302] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114d760b6de344ae8c66ae12ce58ec91, 7c14e55f7ad04080b2531520cb4225c5; no voters:
I20260928 15:57:52.692982 31192 raft_consensus.cc:2781] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:52.693182 31192 raft_consensus.cc:686] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 1 LEADER]: Becoming Leader. State: Replica: 114d760b6de344ae8c66ae12ce58ec91, State: Running, Role: LEADER
I20260928 15:57:52.693517 31192 consensus_queue.cc:237] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.697016 30674 catalog_manager.cc:5495] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "114d760b6de344ae8c66ae12ce58ec91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:52.747984 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.741292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.22.125.194:38697)
0928 15:57:52.741927 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.741933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.741960 (+ 27us) server_negotiation.cc:407] Connection header received
0928 15:57:52.742086 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.742091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.742212 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.742426 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:52.742708 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.743921 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.745327 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.745988 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.746148 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.746518 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.747316 (+ 798us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.747471 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.747542 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.747632 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":332,"threads_started":1}
I20260928 15:57:52.752166 30833 raft_consensus.cc:1261] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Refusing update from remote peer 114d760b6de344ae8c66ae12ce58ec91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:52.752179 31115 raft_consensus.cc:1261] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Refusing update from remote peer 114d760b6de344ae8c66ae12ce58ec91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:52.753051 31192 consensus_queue.cc:1035] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.753435 31242 consensus_queue.cc:1035] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.774339 30674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:52.774667 30674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:52.778388 30674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Sending DeleteTablet for 3 replicas of tablet e0476492ee3c404f883afe1d53c66ae6
I20260928 15:57:52.779294 30952 tablet_service.cc:1514] Processing DeleteTablet for tablet e0476492ee3c404f883afe1d53c66ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53192
I20260928 15:57:52.779300 30813 tablet_service.cc:1514] Processing DeleteTablet for tablet e0476492ee3c404f883afe1d53c66ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:35588
I20260928 15:57:52.779592 31259 tablet_replica.cc:330] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: stopping tablet replica
I20260928 15:57:52.779608 31261 tablet_replica.cc:330] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: stopping tablet replica
I20260928 15:57:52.779754 31259 raft_consensus.cc:2229] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:52.779809 31261 raft_consensus.cc:2229] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:52.780093 31261 raft_consensus.cc:2258] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.780260 31259 raft_consensus.cc:2258] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.781524 31259 ts_tablet_manager.cc:1905] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.781553 31261 ts_tablet_manager.cc:1905] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.788717 31259 ts_tablet_manager.cc:1918] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:52.788880 31259 log.cc:1196] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/e0476492ee3c404f883afe1d53c66ae6
I20260928 15:57:52.788903 31096 tablet_service.cc:1514] Processing DeleteTablet for tablet e0476492ee3c404f883afe1d53c66ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53280
I20260928 15:57:52.789211 31260 tablet_replica.cc:330] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: stopping tablet replica
I20260928 15:57:52.789346 31259 ts_tablet_manager.cc:1939] T e0476492ee3c404f883afe1d53c66ae6 P 114d760b6de344ae8c66ae12ce58ec91: Deleting consensus metadata
I20260928 15:57:52.789426 31260 raft_consensus.cc:2229] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:52.789673 31260 raft_consensus.cc:2258] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.790529 30663 catalog_manager.cc:4841] TS 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697): tablet e0476492ee3c404f883afe1d53c66ae6 (table test-table [id=a0790cfb902746cdb88af83a51267cfa]) successfully deleted
I20260928 15:57:52.791111 31260 ts_tablet_manager.cc:1905] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.791628 31261 ts_tablet_manager.cc:1918] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:52.791815 31261 log.cc:1196] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/e0476492ee3c404f883afe1d53c66ae6
I20260928 15:57:52.792325 31261 ts_tablet_manager.cc:1939] T e0476492ee3c404f883afe1d53c66ae6 P 7c14e55f7ad04080b2531520cb4225c5: Deleting consensus metadata
I20260928 15:57:52.793483 30660 catalog_manager.cc:4841] TS 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099): tablet e0476492ee3c404f883afe1d53c66ae6 (table test-table [id=a0790cfb902746cdb88af83a51267cfa]) successfully deleted
I20260928 15:57:52.794937 30674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:57:52.795830 30674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:57:52.800519 31260 ts_tablet_manager.cc:1918] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:57:52.800693 31260 log.cc:1196] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/e0476492ee3c404f883afe1d53c66ae6
I20260928 15:57:52.801122 31260 ts_tablet_manager.cc:1939] T e0476492ee3c404f883afe1d53c66ae6 P c70bd93245e24bc999fc32991cb03a25: Deleting consensus metadata
I20260928 15:57:52.804349 30813 tablet_service.cc:1467] Processing CreateTablet for tablet 529fca3ef4c04b68bc16d650b7b9186e (DEFAULT_TABLE table=test-table [id=0f0c4353f3994f4595f55245063cbd9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:52.804785 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529fca3ef4c04b68bc16d650b7b9186e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:52.806074 30952 tablet_service.cc:1467] Processing CreateTablet for tablet 529fca3ef4c04b68bc16d650b7b9186e (DEFAULT_TABLE table=test-table [id=0f0c4353f3994f4595f55245063cbd9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:52.806587 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529fca3ef4c04b68bc16d650b7b9186e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:52.811103 31096 tablet_service.cc:1467] Processing CreateTablet for tablet 529fca3ef4c04b68bc16d650b7b9186e (DEFAULT_TABLE table=test-table [id=0f0c4353f3994f4595f55245063cbd9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:57:52.811546 31096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529fca3ef4c04b68bc16d650b7b9186e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:52.812252 30663 catalog_manager.cc:4841] TS c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829): tablet e0476492ee3c404f883afe1d53c66ae6 (table test-table [id=a0790cfb902746cdb88af83a51267cfa]) successfully deleted
I20260928 15:57:52.814306 31275 tablet_bootstrap.cc:492] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: Bootstrap starting.
I20260928 15:57:52.816103 31275 tablet_bootstrap.cc:654] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:52.818300 31277 tablet_bootstrap.cc:492] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: Bootstrap starting.
I20260928 15:57:52.820694 31276 tablet_bootstrap.cc:492] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: Bootstrap starting.
I20260928 15:57:52.820880 31275 tablet_bootstrap.cc:492] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: No bootstrap required, opened a new log
I20260928 15:57:52.821079 31275 ts_tablet_manager.cc:1397] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260928 15:57:52.822068 31276 tablet_bootstrap.cc:654] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:52.822997 31277 tablet_bootstrap.cc:654] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:52.822856 31275 raft_consensus.cc:348] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.823150 31275 raft_consensus.cc:374] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:52.823262 31275 raft_consensus.cc:729] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114d760b6de344ae8c66ae12ce58ec91, State: Initialized, Role: FOLLOWER
I20260928 15:57:52.823567 31275 consensus_queue.cc:260] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.825080 31276 tablet_bootstrap.cc:492] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: No bootstrap required, opened a new log
I20260928 15:57:52.825237 31276 ts_tablet_manager.cc:1397] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260928 15:57:52.825929 31276 raft_consensus.cc:348] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.826201 31276 raft_consensus.cc:374] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:52.826292 31276 raft_consensus.cc:729] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c14e55f7ad04080b2531520cb4225c5, State: Initialized, Role: FOLLOWER
I20260928 15:57:52.826555 31276 consensus_queue.cc:260] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.827806 31275 ts_tablet_manager.cc:1428] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260928 15:57:52.830111 31276 ts_tablet_manager.cc:1428] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260928 15:57:52.832623 31277 tablet_bootstrap.cc:492] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: No bootstrap required, opened a new log
I20260928 15:57:52.832798 31277 ts_tablet_manager.cc:1397] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.004s
I20260928 15:57:52.833527 31277 raft_consensus.cc:348] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.833773 31277 raft_consensus.cc:374] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:52.833847 31277 raft_consensus.cc:729] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70bd93245e24bc999fc32991cb03a25, State: Initialized, Role: FOLLOWER
I20260928 15:57:52.834060 31277 consensus_queue.cc:260] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.834558 31277 ts_tablet_manager.cc:1428] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 15:57:52.853842 31192 raft_consensus.cc:482] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:57:52.854060 31192 raft_consensus.cc:504] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.854795 31192 leader_election.cc:288] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099), c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:52.855301 30833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529fca3ef4c04b68bc16d650b7b9186e" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c14e55f7ad04080b2531520cb4225c5" is_pre_election: true
I20260928 15:57:52.855476 31115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529fca3ef4c04b68bc16d650b7b9186e" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25" is_pre_election: true
I20260928 15:57:52.855561 30833 raft_consensus.cc:2454] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 0.
I20260928 15:57:52.855716 31115 raft_consensus.cc:2454] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 0.
I20260928 15:57:52.856009 30903 leader_election.cc:302] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114d760b6de344ae8c66ae12ce58ec91, 7c14e55f7ad04080b2531520cb4225c5; no voters:
I20260928 15:57:52.856503 31192 raft_consensus.cc:2781] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:57:52.856603 31192 raft_consensus.cc:482] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:57:52.856724 31192 raft_consensus.cc:3037] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.859807 31192 raft_consensus.cc:504] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.860476 31192 leader_election.cc:288] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 election: Requested vote from peers 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099), c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829)
I20260928 15:57:52.860906 30833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529fca3ef4c04b68bc16d650b7b9186e" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c14e55f7ad04080b2531520cb4225c5"
I20260928 15:57:52.861145 30833 raft_consensus.cc:3037] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.861354 31115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529fca3ef4c04b68bc16d650b7b9186e" candidate_uuid: "114d760b6de344ae8c66ae12ce58ec91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70bd93245e24bc999fc32991cb03a25"
I20260928 15:57:52.861553 31115 raft_consensus.cc:3037] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:52.864133 30833 raft_consensus.cc:2454] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 1.
I20260928 15:57:52.864569 30903 leader_election.cc:302] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114d760b6de344ae8c66ae12ce58ec91, 7c14e55f7ad04080b2531520cb4225c5; no voters:
I20260928 15:57:52.864917 31192 raft_consensus.cc:2781] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:52.865412 31192 raft_consensus.cc:686] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 LEADER]: Becoming Leader. State: Replica: 114d760b6de344ae8c66ae12ce58ec91, State: Running, Role: LEADER
I20260928 15:57:52.865682 31192 consensus_queue.cc:237] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } }
I20260928 15:57:52.865852 31115 raft_consensus.cc:2454] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 114d760b6de344ae8c66ae12ce58ec91 in term 1.
I20260928 15:57:52.868983 30674 catalog_manager.cc:5495] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "114d760b6de344ae8c66ae12ce58ec91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "114d760b6de344ae8c66ae12ce58ec91" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 38697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260928 15:57:52.910007 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.902295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39099 (local address 127.22.125.194:42545)
0928 15:57:52.904525 (+ 2230us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:52.904546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:52.904702 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:52.905073 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:52.905076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:52.905087 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:52.905383 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.905394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.906809 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.906813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.907953 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.907964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.908976 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.908981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.909092 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.909144 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.909540 (+ 396us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.909548 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 15:57:52.909605 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 15:57:52.909700 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2021,"thread_start_us":1900,"threads_started":1}
I20260928 15:57:52.910074 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.902209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:42545 (local address 127.22.125.193:39099)
0928 15:57:52.902700 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.902705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.904644 (+ 1939us) server_negotiation.cc:407] Connection header received
0928 15:57:52.904853 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.904859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.904951 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.905060 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:52.905544 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.906682 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.908109 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.908797 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.908926 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.909318 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.909448 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.909534 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.909655 (+ 121us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.909749 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":259,"threads_started":1}
I20260928 15:57:52.918469 31116 raft_consensus.cc:1261] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Refusing update from remote peer 114d760b6de344ae8c66ae12ce58ec91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:52.918473 30833 raft_consensus.cc:1261] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Refusing update from remote peer 114d760b6de344ae8c66ae12ce58ec91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:57:52.919021 31242 consensus_queue.cc:1035] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70bd93245e24bc999fc32991cb03a25" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.920194 31242 consensus_queue.cc:1035] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c14e55f7ad04080b2531520cb4225c5" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:57:52.947134 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.941243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:37553 (local address 127.22.125.194:38697)
0928 15:57:52.941459 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.941465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:52.941846 (+ 381us) server_negotiation.cc:407] Connection header received
0928 15:57:52.942031 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:52.942035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:52.942124 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:52.942223 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:57:52.942669 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.943764 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.945254 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.945696 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.945874 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.946425 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.946539 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.946645 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:52.946788 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 15:57:52.946863 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 15:57:52.947310 31301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.941163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:38697 (local address 127.22.125.193:37553)
0928 15:57:52.941733 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:52.941758 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:52.941895 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:52.942254 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:52.942259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:52.942272 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 15:57:52.942516 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.942527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.943905 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.943910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.945111 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:52.945123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:52.946005 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:52.946010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:52.946207 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:52.946259 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 15:57:52.946653 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 15:57:52.946675 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 15:57:52.946760 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 15:57:52.946901 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":271,"threads_started":1}
I20260928 15:57:52.980259 30676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:52.980558 30676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:57:52.983505 30676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52d8a9b460a04157b1ab83500c1a2c8d: Sending DeleteTablet for 3 replicas of tablet 529fca3ef4c04b68bc16d650b7b9186e
I20260928 15:57:52.984169 30813 tablet_service.cc:1514] Processing DeleteTablet for tablet 529fca3ef4c04b68bc16d650b7b9186e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:35588
I20260928 15:57:52.984381 31096 tablet_service.cc:1514] Processing DeleteTablet for tablet 529fca3ef4c04b68bc16d650b7b9186e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53280
I20260928 15:57:52.984477 30952 tablet_service.cc:1514] Processing DeleteTablet for tablet 529fca3ef4c04b68bc16d650b7b9186e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:57:52 UTC) from {username='oryx', principal='oryx/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:53192
I20260928 15:57:52.984706 31261 tablet_replica.cc:330] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: stopping tablet replica
I20260928 15:57:52.984762 31259 tablet_replica.cc:330] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: stopping tablet replica
I20260928 15:57:52.984923 31261 raft_consensus.cc:2229] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:52.984933 31259 raft_consensus.cc:2229] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:57:52.985142 31261 raft_consensus.cc:2258] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.985285 31259 raft_consensus.cc:2258] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.985438 31260 tablet_replica.cc:330] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: stopping tablet replica
I20260928 15:57:52.985628 31260 raft_consensus.cc:2229] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:57:52.985852 31260 raft_consensus.cc:2258] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:57:52.986349 31261 ts_tablet_manager.cc:1905] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.986853 31259 ts_tablet_manager.cc:1905] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.991583 31259 ts_tablet_manager.cc:1918] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:52.991742 31259 log.cc:1196] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/529fca3ef4c04b68bc16d650b7b9186e
I20260928 15:57:52.992246 31259 ts_tablet_manager.cc:1939] T 529fca3ef4c04b68bc16d650b7b9186e P 114d760b6de344ae8c66ae12ce58ec91: Deleting consensus metadata
I20260928 15:57:52.992388 31260 ts_tablet_manager.cc:1905] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:57:52.992965 31261 ts_tablet_manager.cc:1918] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:52.993083 31261 log.cc:1196] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/529fca3ef4c04b68bc16d650b7b9186e
I20260928 15:57:52.993261 30663 catalog_manager.cc:4841] TS 114d760b6de344ae8c66ae12ce58ec91 (127.22.125.194:38697): tablet 529fca3ef4c04b68bc16d650b7b9186e (table test-table [id=0f0c4353f3994f4595f55245063cbd9f]) successfully deleted
I20260928 15:57:52.994716 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 30747
I20260928 15:57:52.996402 31261 ts_tablet_manager.cc:1939] T 529fca3ef4c04b68bc16d650b7b9186e P 7c14e55f7ad04080b2531520cb4225c5: Deleting consensus metadata
I20260928 15:57:52.997686 30660 catalog_manager.cc:4841] TS 7c14e55f7ad04080b2531520cb4225c5 (127.22.125.193:39099): tablet 529fca3ef4c04b68bc16d650b7b9186e (table test-table [id=0f0c4353f3994f4595f55245063cbd9f]) successfully deleted
I20260928 15:57:53.000727 31260 ts_tablet_manager.cc:1918] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 15:57:53.000876 31260 log.cc:1196] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/529fca3ef4c04b68bc16d650b7b9186e
I20260928 15:57:53.001334 31260 ts_tablet_manager.cc:1939] T 529fca3ef4c04b68bc16d650b7b9186e P c70bd93245e24bc999fc32991cb03a25: Deleting consensus metadata
I20260928 15:57:53.002439 30663 catalog_manager.cc:4841] TS c70bd93245e24bc999fc32991cb03a25 (127.22.125.195:34829): tablet 529fca3ef4c04b68bc16d650b7b9186e (table test-table [id=0f0c4353f3994f4595f55245063cbd9f]) successfully deleted
Sep 28 15:57:53 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.193@KRBTEST.COM
I20260928 15:57:53.015597 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:52.997024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.22.125.193:39099)
0928 15:57:52.997182 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:52.997189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:53.000529 (+ 3340us) server_negotiation.cc:407] Connection header received
0928 15:57:53.000698 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:53.000703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:53.000788 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:53.000897 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:53.002710 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:53.003416 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:53.004712 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:53.005135 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:53.005267 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:53.009212 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:53.009226 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:53.009242 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:53.009280 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:53.012928 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:53.013579 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:53.013589 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:53.013593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:53.013771 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:53.014099 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:53.014105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:53.014108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:53.014549 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:53.014865 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:53.015016 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 15:57:53.015240 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 15:57:53.016378 30878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:53.200107 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 30747
I20260928 15:57:53.235980 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 30887
Sep 28 15:57:53 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.194@KRBTEST.COM
I20260928 15:57:53.253417 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:53.236858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46928 (local address 127.22.125.194:38697)
0928 15:57:53.237057 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:53.237065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:53.237090 (+ 25us) server_negotiation.cc:407] Connection header received
0928 15:57:53.237178 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:53.237184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:53.237281 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:53.237417 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:53.238694 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:53.239859 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:53.241455 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:53.242139 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:53.242279 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:53.246767 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:53.246779 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:53.246798 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:53.246839 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:53.250732 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:53.251344 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:53.251353 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:53.251356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:53.251499 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:53.251819 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:53.251824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:53.251828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:53.252273 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:53.252600 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:53.252797 (+ 197us) server_negotiation.cc:299] Negotiation successful
0928 15:57:53.253009 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 15:57:53.254143 31017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:53.393110 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 30887
I20260928 15:57:53.415653 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31026
Sep 28 15:57:53 dist-test-slave-9cqd krb5kdc[30631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.125.195@KRBTEST.COM
I20260928 15:57:53.430133 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:53.416489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.22.125.195:34829)
0928 15:57:53.416990 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:53.416995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:53.417010 (+ 15us) server_negotiation.cc:407] Connection header received
0928 15:57:53.417065 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:53.417069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:53.417134 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:53.417224 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:53.418446 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:53.419116 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:53.420407 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:53.420813 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:53.420905 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:53.424983 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:53.424997 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:53.425011 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:53.425042 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:53.427917 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:53.428507 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:53.428514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:53.428516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:53.428636 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:53.428971 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:53.428974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:53.428976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:53.429288 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:53.429523 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:53.429713 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 15:57:53.429879 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":301,"threads_started":1}
I20260928 15:57:53.430873 31156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:53.556478 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 31026
I20260928 15:57:53.577827 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 30644
I20260928 15:57:53.579025 30716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:53.694600 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 30644
2026-09-28T15:57:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5917 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:53 dist-test-slave-9cqd krb5kdc[31312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:53 dist-test-slave-9cqd krb5kdc[31312](info): set up 2 sockets
Sep 28 15:57:53 dist-test-slave-9cqd krb5kdc[31312](info): commencing operation
krb5kdc: starting...
W20260928 15:57:54.787133 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:54 dist-test-slave-9cqd krb5kdc[31312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:54.995607 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34117
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:34117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:55.311566 31325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:55.311915 31325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:55.312105 31325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:55.319983 31325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:55.320158 31325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:55.320266 31325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:55.320349 31325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:55.320452 31325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:55.330862 31325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:34117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:55.332219 31325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:55.333551 31325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:55.345077 31331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:55.345284 31330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:55.345216 31333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:55.346230 31325 server_base.cc:1034] running on GCE node
I20260928 15:57:55.347018 31325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:55.348443 31325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:55.349646 31325 hybrid_clock.cc:648] HybridClock initialized: now 1790611075349604 us; error 32 us; skew 500 ppm
I20260928 15:57:55.349993 31325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T15:57:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1725 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:57:55 dist-test-slave-9cqd krb5kdc[31339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:57:55 dist-test-slave-9cqd krb5kdc[31339](info): set up 2 sockets
Sep 28 15:57:55 dist-test-slave-9cqd krb5kdc[31339](info): commencing operation
krb5kdc: starting...
W20260928 15:57:56.510968 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:57:56 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:57:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:57:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:57:56.719830 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:38121
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:38121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:57.039197 31352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:57.039547 31352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:57.039712 31352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:57.046957 31352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:57.047099 31352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:57.047200 31352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:57.047304 31352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:57.047379 31352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:57.057217 31352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:38121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:38121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:57.058517 31352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:57.059810 31352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:57.071835 31357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:57.071792 31360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:57.071928 31358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:57.072539 31352 server_base.cc:1034] running on GCE node
I20260928 15:57:57.073319 31352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:57.074656 31352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:57.075884 31352 hybrid_clock.cc:648] HybridClock initialized: now 1790611077075848 us; error 51 us; skew 500 ppm
I20260928 15:57:57.076283 31352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:57 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611077, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:57.079214 31352 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:57.080757 31352 webserver.cc:460] Webserver started at http://127.22.125.254:40377/ using document root <none> and password file <none>
I20260928 15:57:57.081434 31352 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:57.081578 31352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:57.081887 31352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:57.084789 31352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "5f7cc5c259844a07a395fb32b53885ff"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "ee64f614eebc5e453332a03737a5d04f"
server_key_iv: "7fa71273135dc62fd031320bf635be50"
server_key_version: "encryptionkey@0"
I20260928 15:57:57.085530 31352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "5f7cc5c259844a07a395fb32b53885ff"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "ee64f614eebc5e453332a03737a5d04f"
server_key_iv: "7fa71273135dc62fd031320bf635be50"
server_key_version: "encryptionkey@0"
I20260928 15:57:57.090528 31352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:57.093789 31367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:57.094960 31352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 15:57:57.095147 31352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "5f7cc5c259844a07a395fb32b53885ff"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "ee64f614eebc5e453332a03737a5d04f"
server_key_iv: "7fa71273135dc62fd031320bf635be50"
server_key_version: "encryptionkey@0"
I20260928 15:57:57.095346 31352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:57.114614 31352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:57.118500 31352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:57.118788 31352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:57.137182 31352 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:38121
I20260928 15:57:57.137212 31418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:38121 every 8 connection(s)
I20260928 15:57:57.138525 31352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:57.139043 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 31352
I20260928 15:57:57.139292 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:57:57.140153 23031 external_mini_cluster.cc:1428] Setting key c44edc3ec496746f19188a1d1d8ffa65
I20260928 15:57:57.142867 31419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:57:57.153054 31419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Bootstrap starting.
Sep 28 15:57:57 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:57.156306 31419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Neither blocks nor log segments found. Creating new log.
I20260928 15:57:57.157300 31419 log.cc:824] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:57.160545 31419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: No bootstrap required, opened a new log
I20260928 15:57:57.162070 31422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:57.142059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39060 (local address 127.22.125.254:38121)
0928 15:57:57.142809 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:57.142820 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:57.142866 (+ 46us) server_negotiation.cc:407] Connection header received
0928 15:57:57.144553 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:57.144574 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:57.144919 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:57.145305 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:57.146950 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:57.148345 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:57.149546 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:57.150202 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:57.150434 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:57.154343 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:57.154351 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:57.154373 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:57.154409 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:57.157557 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:57.158395 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:57.158401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:57.158406 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:57.158625 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:57.159009 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:57.159013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:57.159017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:57.159646 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:57.159970 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:57.160317 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 15:57:57.160728 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":354,"threads_started":1}
I20260928 15:57:57.168893 31419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:57.169204 31419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:57:57.169317 31419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7cc5c259844a07a395fb32b53885ff, State: Initialized, Role: FOLLOWER
I20260928 15:57:57.169872 31419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:57.170078 31419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:57.170172 31419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:57.170301 31419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:57:57.172775 31419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:57.173203 31419 leader_election.cc:302] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f7cc5c259844a07a395fb32b53885ff; no voters:
I20260928 15:57:57.173736 31419 leader_election.cc:288] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:57:57.173874 31424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:57:57.174587 31424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 LEADER]: Becoming Leader. State: Replica: 5f7cc5c259844a07a395fb32b53885ff, State: Running, Role: LEADER
I20260928 15:57:57.175093 31424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:57.175714 31419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:57.177732 31425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f7cc5c259844a07a395fb32b53885ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } } }
I20260928 15:57:57.178138 31425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:57.178700 31426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f7cc5c259844a07a395fb32b53885ff. Latest consensus state: current_term: 1 leader_uuid: "5f7cc5c259844a07a395fb32b53885ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } } }
I20260928 15:57:57.179067 31426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:57.182760 31429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:57.188671 31429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:57.197571 31429 catalog_manager.cc:1347] Generated new cluster ID: bee03936ba904c9a9f175b7e93c7d829
I20260928 15:57:57.197706 31429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:57.211779 31429 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:57:57.213557 31429 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:57.237308 31429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Generated new TSK 0
I20260928 15:57:57.238333 31429 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:57:57.291630 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:38121
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:57.607964 31446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:57.608316 31446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:57.608415 31446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:57.615542 31446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:57.615648 31446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:57.615758 31446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:57:57.625310 31446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:38121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:57.626528 31446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:57.627794 31446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:57.640717 31451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:57.640682 31454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:57.640746 31452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:57.641706 31446 server_base.cc:1034] running on GCE node
I20260928 15:57:57.642302 31446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:57.643196 31446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:57.644421 31446 hybrid_clock.cc:648] HybridClock initialized: now 1790611077644373 us; error 41 us; skew 500 ppm
I20260928 15:57:57.644730 31446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:57 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611077, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:57.647871 31446 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:57:57.649351 31446 webserver.cc:460] Webserver started at http://127.22.125.193:45341/ using document root <none> and password file <none>
I20260928 15:57:57.650046 31446 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:57.650194 31446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:57.650501 31446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:57.653399 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "2a547e97e4c048f49ff1aeb13d128210"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "a2691fc0e12cd895698095fec90a0251"
server_key_iv: "022bda8b6a323126ec6d86aed3804b7d"
server_key_version: "encryptionkey@0"
I20260928 15:57:57.654170 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "2a547e97e4c048f49ff1aeb13d128210"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "a2691fc0e12cd895698095fec90a0251"
server_key_iv: "022bda8b6a323126ec6d86aed3804b7d"
server_key_version: "encryptionkey@0"
I20260928 15:57:57.659428 31446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 15:57:57.663021 31461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:57.664402 31446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 15:57:57.664573 31446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "2a547e97e4c048f49ff1aeb13d128210"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "a2691fc0e12cd895698095fec90a0251"
server_key_iv: "022bda8b6a323126ec6d86aed3804b7d"
server_key_version: "encryptionkey@0"
I20260928 15:57:57.664768 31446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:57.678745 31446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:57.683203 31446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:57.683499 31446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:57.684520 31446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:57.685796 31446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:57.685897 31446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:57.686043 31446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:57.686128 31446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:57.718807 31446 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:46203
I20260928 15:57:57.718858 31573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:46203 every 8 connection(s)
I20260928 15:57:57.720144 31446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:57:57.721683 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 31446
I20260928 15:57:57.721894 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:57:57.722683 23031 external_mini_cluster.cc:1428] Setting key 884335eacb06f2bf43aabfd4e320287b
Sep 28 15:57:57 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611077, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:57.741601 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:57.722937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:44405 (local address 127.22.125.254:38121)
0928 15:57:57.723392 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:57.723397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:57.724466 (+ 1069us) server_negotiation.cc:407] Connection header received
0928 15:57:57.726395 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:57.726404 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:57.726499 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:57.726622 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:57.728109 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:57.729248 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:57.730753 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:57.731534 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:57.731678 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:57.735763 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:57.735768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:57.735771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:57.735803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:57.738515 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:57.739222 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:57.739229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:57.739232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:57.739363 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:57.739796 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:57.739801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:57.739815 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:57.740171 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:57.740328 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:57.741001 (+ 673us) server_negotiation.cc:299] Negotiation successful
0928 15:57:57.741223 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":242,"threads_started":1}
I20260928 15:57:57.742518 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:57.723528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:44405)
0928 15:57:57.724267 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:57.724348 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:57.726128 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:57.726829 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:57.726837 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:57.727345 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:57.727911 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:57.727930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:57.729397 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:57.729401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:57.730605 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:57.730616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:57.731734 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:57.731739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:57.732132 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:57.732890 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:57.732923 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:57.735551 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:57.738655 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:57.738664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:57.738680 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:57.739101 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:57.739485 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:57.739490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:57.739494 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:57.739679 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:57.740297 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:57.740305 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:57.740645 (+ 340us) client_negotiation.cc:769] Sending connection context
0928 15:57:57.740914 (+ 269us) client_negotiation.cc:240] Negotiation successful
0928 15:57:57.741428 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":473,"threads_started":1}
I20260928 15:57:57.744130 31574 heartbeater.cc:347] Connected to a master server at 127.22.125.254:38121
I20260928 15:57:57.744438 31574 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:57:57.745154 31574 heartbeater.cc:511] Master 127.22.125.254:38121 requested a full tablet report, sending...
I20260928 15:57:57.747449 31384 ts_manager.cc:194] Registered new tserver with Master: 2a547e97e4c048f49ff1aeb13d128210 (127.22.125.193:46203)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:57:57.749049 31384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:44405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:57:57.773196 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:38121
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:58.092185 31580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:58.092453 31580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:58.092546 31580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:58.099628 31580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:58.099731 31580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:58.099841 31580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:57:58.109675 31580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:38121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:58.110890 31580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:58.112265 31580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:58.125068 31588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:58.125133 31586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:58.125216 31585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:58.125842 31580 server_base.cc:1034] running on GCE node
I20260928 15:57:58.126453 31580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:58.127293 31580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:58.128484 31580 hybrid_clock.cc:648] HybridClock initialized: now 1790611078128452 us; error 27 us; skew 500 ppm
I20260928 15:57:58.128818 31580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:58 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611078, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:58.131947 31580 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:57:58.133440 31580 webserver.cc:460] Webserver started at http://127.22.125.194:45991/ using document root <none> and password file <none>
I20260928 15:57:58.134115 31580 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:58.134246 31580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:58.134570 31580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:58.137511 31580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "f10d5bc8d5d14527ae31dfcc28ce6511"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "5deffd2f43cfe2debf2e4989eca91bc5"
server_key_iv: "1ad7b800292ec1efbe4e7bff3a2c93b2"
server_key_version: "encryptionkey@0"
I20260928 15:57:58.138291 31580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "f10d5bc8d5d14527ae31dfcc28ce6511"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "5deffd2f43cfe2debf2e4989eca91bc5"
server_key_iv: "1ad7b800292ec1efbe4e7bff3a2c93b2"
server_key_version: "encryptionkey@0"
I20260928 15:57:58.143486 31580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 15:57:58.146973 31595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:58.148406 31580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 15:57:58.148577 31580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "f10d5bc8d5d14527ae31dfcc28ce6511"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "5deffd2f43cfe2debf2e4989eca91bc5"
server_key_iv: "1ad7b800292ec1efbe4e7bff3a2c93b2"
server_key_version: "encryptionkey@0"
I20260928 15:57:58.148759 31580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:58.160741 31580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:58.165146 31580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:58.165450 31580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:58.166373 31580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:58.167652 31580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:58.167757 31580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:58.167899 31580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:58.167976 31580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:58.201169 31580 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:42743
I20260928 15:57:58.201220 31707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:42743 every 8 connection(s)
I20260928 15:57:58.202525 31580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:57:58.203647 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 31580
I20260928 15:57:58.203896 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:57:58.204521 23031 external_mini_cluster.cc:1428] Setting key 77c5d70569e5c8f4950463a3c68331ef
Sep 28 15:57:58 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611078, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:58.224661 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:58.205248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:54561 (local address 127.22.125.254:38121)
0928 15:57:58.205447 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:58.205451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:58.206664 (+ 1213us) server_negotiation.cc:407] Connection header received
0928 15:57:58.208546 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:58.208556 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:58.208652 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:58.208768 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:58.210150 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.211325 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.213136 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.213880 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.214030 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:58.217533 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:58.217542 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:58.217550 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:58.217594 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:58.221289 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.222064 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.222073 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.222080 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.222245 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.222656 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.222664 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.222670 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.223053 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:58.223231 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:58.223885 (+ 654us) server_negotiation.cc:299] Negotiation successful
0928 15:57:58.224186 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 15:57:58.225232 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:58.205674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:54561)
0928 15:57:58.206514 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:58.206567 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:58.208282 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:58.208949 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:58.208957 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:58.209442 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:58.209982 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:58.209999 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.211543 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.211552 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:58.212989 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:58.213002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.214076 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.214081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:58.214268 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:58.214953 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:58.214974 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:58.217304 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:58.221423 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:58.221429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:58.221440 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:58.221925 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:58.222358 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:58.222362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:58.222364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:58.222529 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:58.223199 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:58.223207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:58.223510 (+ 303us) client_negotiation.cc:769] Sending connection context
0928 15:57:58.223725 (+ 215us) client_negotiation.cc:240] Negotiation successful
0928 15:57:58.224150 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":355,"threads_started":1}
I20260928 15:57:58.226756 31708 heartbeater.cc:347] Connected to a master server at 127.22.125.254:38121
I20260928 15:57:58.227064 31708 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 15:57:58.227778 31708 heartbeater.cc:511] Master 127.22.125.254:38121 requested a full tablet report, sending...
I20260928 15:57:58.229413 31384 ts_manager.cc:194] Registered new tserver with Master: f10d5bc8d5d14527ae31dfcc28ce6511 (127.22.125.194:42743)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
I20260928 15:57:58.230428 31384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:54561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:57:58.253206 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:38121
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:58.573992 31713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:58.574301 31713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:58.574467 31713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:58.581768 31713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:58.581923 31713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:58.582088 31713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:57:58.591459 31713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:38121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:58.592790 31713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:58.594094 31713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:58.607064 31718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:58.607035 31719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:58.607125 31721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:58.608078 31713 server_base.cc:1034] running on GCE node
I20260928 15:57:58.608661 31713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:58.609543 31713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:58.610767 31713 hybrid_clock.cc:648] HybridClock initialized: now 1790611078610723 us; error 58 us; skew 500 ppm
I20260928 15:57:58.611321 31713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:58 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611078, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:58.614461 31713 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:57:58.615871 31713 webserver.cc:460] Webserver started at http://127.22.125.195:42231/ using document root <none> and password file <none>
I20260928 15:57:58.616675 31713 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:58.616819 31713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:58.617116 31713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:57:58.619901 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "42136caf4bb743fabf91fc3f5475a648"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "3aab1054bdc3669ebb6b84f919e0b996"
server_key_iv: "9153cddbb927ccd480beca01bce87158"
server_key_version: "encryptionkey@0"
I20260928 15:57:58.620749 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "42136caf4bb743fabf91fc3f5475a648"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "3aab1054bdc3669ebb6b84f919e0b996"
server_key_iv: "9153cddbb927ccd480beca01bce87158"
server_key_version: "encryptionkey@0"
I20260928 15:57:58.625792 31713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:57:58.629046 31729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:58.630307 31713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:57:58.630496 31713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "42136caf4bb743fabf91fc3f5475a648"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "3aab1054bdc3669ebb6b84f919e0b996"
server_key_iv: "9153cddbb927ccd480beca01bce87158"
server_key_version: "encryptionkey@0"
I20260928 15:57:58.630685 31713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:58.646808 31713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:58.650929 31713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:58.651202 31713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:58.652305 31713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:57:58.653584 31713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:57:58.653685 31713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:58.653841 31713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:57:58.653913 31713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:58.686611 31713 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:42629
I20260928 15:57:58.686652 31841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:42629 every 8 connection(s)
I20260928 15:57:58.687965 31713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:57:58.692625 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 31713
I20260928 15:57:58.692873 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:57:58.693387 23031 external_mini_cluster.cc:1428] Setting key 10813a7e97e94cb49141aed333ca93bc
Sep 28 15:57:58 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611078, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:57:58.707232 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:58.690532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:41459 (local address 127.22.125.254:38121)
0928 15:57:58.690703 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:58.690707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:58.691844 (+ 1137us) server_negotiation.cc:407] Connection header received
0928 15:57:58.693872 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:58.693878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:58.693976 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:58.694085 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:58.695520 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.696345 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.697862 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.698267 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.698428 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:58.701803 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:58.701808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:58.701811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:58.701842 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:58.704273 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.704972 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.704979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.704983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.705107 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.705550 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.705556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.705560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.705835 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:58.705998 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:58.706658 (+ 660us) server_negotiation.cc:299] Negotiation successful
0928 15:57:58.706881 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 15:57:58.708350 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:58.690961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:41459)
0928 15:57:58.691681 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0928 15:57:58.691724 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 15:57:58.693605 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:57:58.694319 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:57:58.694327 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:57:58.694802 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:57:58.695351 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:58.695369 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.696510 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.696514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:58.697709 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:57:58.697721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.698532 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.698540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:57:58.698782 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:58.699417 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:57:58.699439 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:57:58.701579 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:57:58.704439 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:58.704447 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:58.704461 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:58.704843 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:58.705221 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:57:58.705226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:57:58.705229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:57:58.705421 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:57:58.705967 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:57:58.705974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:57:58.706299 (+ 325us) client_negotiation.cc:769] Sending connection context
0928 15:57:58.706562 (+ 263us) client_negotiation.cc:240] Negotiation successful
0928 15:57:58.707057 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":283,"threads_started":1}
I20260928 15:57:58.709664 31842 heartbeater.cc:347] Connected to a master server at 127.22.125.254:38121
I20260928 15:57:58.709954 31842 heartbeater.cc:464] Registering TS with master...
I20260928 15:57:58.710826 31842 heartbeater.cc:511] Master 127.22.125.254:38121 requested a full tablet report, sending...
I20260928 15:57:58.712469 31384 ts_manager.cc:194] Registered new tserver with Master: 42136caf4bb743fabf91fc3f5475a648 (127.22.125.195:42629)
I20260928 15:57:58.713692 31384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:41459
I20260928 15:57:58.720878 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.193@KRBTEST.COM" created.
I20260928 15:57:58.751837 31574 heartbeater.cc:503] Master 127.22.125.254:38121 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab.
Entry for principal oryx/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab.
WARNING: no policy specified for oryx/127.22.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab.
Entry for principal oryx/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab.
WARNING: no policy specified for oryx/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab.
Entry for principal oryx/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab.
I20260928 15:57:58.847057 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31446
Sep 28 15:57:58 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:57:58.863201 31852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:58.848248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56542 (local address 127.22.125.193:46203)
0928 15:57:58.848913 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:58.848923 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:58.848957 (+ 34us) server_negotiation.cc:407] Connection header received
0928 15:57:58.849071 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:58.849080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:58.849319 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:58.849510 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:58.850866 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.851932 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.853258 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.853918 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.854059 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:58.858087 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:58.858098 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:58.858112 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:58.858153 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:58.860851 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.861457 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.861464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.861466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.861586 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.861900 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.861903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.861905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.862216 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:58.862437 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:58.862726 (+ 289us) server_negotiation.cc:299] Negotiation successful
0928 15:57:58.862920 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":291,"threads_started":1}
I20260928 15:57:58.864552 31569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:58.947217 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 31446
I20260928 15:57:58.963639 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31580
Sep 28 15:57:58 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:57:58.979957 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:58.964584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.22.125.194:42743)
0928 15:57:58.965147 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:58.965155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:58.965179 (+ 24us) server_negotiation.cc:407] Connection header received
0928 15:57:58.965238 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:58.965245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:58.965466 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:58.965642 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:58.966911 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.967939 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.969347 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:58.969919 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:58.970065 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:58.974330 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:58.974341 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:58.974361 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:58.974396 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:58.977186 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.977842 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.977850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.977852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.977972 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:58.978345 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:58.978351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:58.978355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:58.978686 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:58.979010 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:58.979327 (+ 317us) server_negotiation.cc:299] Negotiation successful
0928 15:57:58.979611 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":282,"threads_started":1}
I20260928 15:57:58.981251 31703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:59.063688 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 31580
I20260928 15:57:59.079537 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31713
Sep 28 15:57:59 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:57:59.096546 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:57:59.080657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.22.125.195:42629)
0928 15:57:59.081319 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0928 15:57:59.081326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:57:59.081362 (+ 36us) server_negotiation.cc:407] Connection header received
0928 15:57:59.081428 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:57:59.081434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:57:59.081643 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:57:59.081826 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:57:59.083210 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:59.084365 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:59.085698 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:57:59.086262 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:57:59.086423 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:57:59.090713 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:57:59.090724 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:57:59.090738 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:57:59.090772 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:57:59.093732 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:59.094416 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:59.094423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:59.094426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:59.094552 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:57:59.094928 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:57:59.094935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:57:59.094938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:57:59.095276 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:57:59.095611 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0928 15:57:59.095903 (+ 292us) server_negotiation.cc:299] Negotiation successful
0928 15:57:59.096196 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":353,"threads_started":1}
I20260928 15:57:59.097818 31837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:59.186594 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 31713
I20260928 15:57:59.202401 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31352
I20260928 15:57:59.203644 31414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:57:59.307875 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 31352
I20260928 15:57:59.328231 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:38121
--webserver_interface=127.22.125.254
--webserver_port=40377
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:38121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:57:59.645680 31859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:57:59.646008 31859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:57:59.646173 31859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:57:59.653476 31859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:57:59.653609 31859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:57:59.653707 31859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:57:59.653807 31859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:57:59.653878 31859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:57:59.663645 31859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:38121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:38121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=40377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:57:59.664947 31859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:57:59.666265 31859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:57:59.678042 31867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:59.678205 31865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:57:59.678413 31864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:57:59.679019 31859 server_base.cc:1034] running on GCE node
I20260928 15:57:59.679806 31859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:57:59.681201 31859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:57:59.682440 31859 hybrid_clock.cc:648] HybridClock initialized: now 1790611079682402 us; error 52 us; skew 500 ppm
I20260928 15:57:59.682792 31859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:57:59 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611079, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:57:59.686246 31859 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:57:59.687620 31859 webserver.cc:460] Webserver started at http://127.22.125.254:40377/ using document root <none> and password file <none>
I20260928 15:57:59.688397 31859 fs_manager.cc:359] Metadata directory not provided
I20260928 15:57:59.688510 31859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:57:59.693737 31859 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 15:57:59.696444 31874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:57:59.697855 31859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 15:57:59.697997 31859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "5f7cc5c259844a07a395fb32b53885ff"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "ee64f614eebc5e453332a03737a5d04f"
server_key_iv: "7fa71273135dc62fd031320bf635be50"
server_key_version: "encryptionkey@0"
I20260928 15:57:59.699252 31859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:57:59.731699 31859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:57:59.735644 31859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:57:59.735860 31859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:57:59.754323 31859 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:38121
I20260928 15:57:59.754343 31925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:38121 every 8 connection(s)
I20260928 15:57:59.755774 31859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:57:59.759490 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 31859
I20260928 15:57:59.760351 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:46203
--local_ip_for_outbound_sockets=127.22.125.193
--tserver_master_addrs=127.22.125.254:38121
--webserver_port=45341
--webserver_interface=127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 15:57:59.762693 31926 sys_catalog.cc:263] Verifying existing consensus state
I20260928 15:57:59.767202 31926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Bootstrap starting.
I20260928 15:57:59.785899 31926 log.cc:824] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Log is configured to *not* fsync() on all Append() calls
I20260928 15:57:59.794350 31926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 15:57:59.794868 31926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Bootstrap complete.
I20260928 15:57:59.803890 31926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:59.804792 31926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7cc5c259844a07a395fb32b53885ff, State: Initialized, Role: FOLLOWER
I20260928 15:57:59.805378 31926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:59.805632 31926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:57:59.805754 31926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:57:59.805941 31926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 1 FOLLOWER]: Advancing to term 2
I20260928 15:57:59.809554 31926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:59.810027 31926 leader_election.cc:302] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f7cc5c259844a07a395fb32b53885ff; no voters:
I20260928 15:57:59.810678 31926 leader_election.cc:288] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 15:57:59.810812 31930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 2 FOLLOWER]: Leader election won for term 2
I20260928 15:57:59.812125 31930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [term 2 LEADER]: Becoming Leader. State: Replica: 5f7cc5c259844a07a395fb32b53885ff, State: Running, Role: LEADER
I20260928 15:57:59.812721 31930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } }
I20260928 15:57:59.812999 31926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:57:59.815263 31931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5f7cc5c259844a07a395fb32b53885ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } } }
I20260928 15:57:59.815711 31931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:59.820703 31932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f7cc5c259844a07a395fb32b53885ff. Latest consensus state: current_term: 2 leader_uuid: "5f7cc5c259844a07a395fb32b53885ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f7cc5c259844a07a395fb32b53885ff" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 38121 } } }
I20260928 15:57:59.821070 31932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff [sys.catalog]: This master's current role is: LEADER
I20260928 15:57:59.823984 31936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:57:59.831866 31936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:57:59.833281 31936 catalog_manager.cc:1259] Loaded cluster ID: bee03936ba904c9a9f175b7e93c7d829
I20260928 15:57:59.833423 31936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:57:59.837420 31936 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:57:59.838845 31936 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f7cc5c259844a07a395fb32b53885ff: Loaded TSK: 0
I20260928 15:57:59.840333 31936 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 15:58:00.093926 31928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:00.094185 31928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:00.094354 31928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:00.101707 31928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:00.101816 31928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:00.101989 31928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:58:00.111999 31928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.193.keytab
--principal=oryx/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:46203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=45341
--tserver_master_addrs=127.22.125.254:38121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:00.113422 31928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:00.114769 31928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:00.129447 31953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:00.129704 31955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:00.130007 31952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:00.130383 31928 server_base.cc:1034] running on GCE node
I20260928 15:58:00.130965 31928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:00.131949 31928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:00.133163 31928 hybrid_clock.cc:648] HybridClock initialized: now 1790611080133112 us; error 46 us; skew 500 ppm
I20260928 15:58:00.133476 31928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611080, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:58:00.136693 31928 init.cc:377] Logged in from keytab as oryx/127.22.125.193@KRBTEST.COM (short username oryx)
I20260928 15:58:00.138162 31928 webserver.cc:460] Webserver started at http://127.22.125.193:45341/ using document root <none> and password file <none>
I20260928 15:58:00.138837 31928 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:00.138994 31928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:00.144420 31928 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:58:00.147110 31962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:00.148520 31928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:58:00.148686 31928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "2a547e97e4c048f49ff1aeb13d128210"
format_stamp: "Formatted at 2026-09-28 15:57:57 on dist-test-slave-9cqd"
server_key: "a2691fc0e12cd895698095fec90a0251"
server_key_iv: "022bda8b6a323126ec6d86aed3804b7d"
server_key_version: "encryptionkey@0"
I20260928 15:58:00.149609 31928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:00.165271 31928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:00.169250 31928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:00.169523 31928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:00.170440 31928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:00.171836 31928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:00.171962 31928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:00.172113 31928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:00.172214 31928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:00.204818 31928 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:46203
I20260928 15:58:00.204879 32074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:46203 every 8 connection(s)
I20260928 15:58:00.206319 31928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:58:00.213013 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 31928
I20260928 15:58:00.213781 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:42743
--local_ip_for_outbound_sockets=127.22.125.194
--tserver_master_addrs=127.22.125.254:38121
--webserver_port=45991
--webserver_interface=127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:00.226768 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:00.209260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:37321)
0928 15:58:00.210056 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:00.210098 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:00.211760 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:00.213401 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:00.213408 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:00.214124 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:00.214717 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.214736 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.217033 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.217037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.218227 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.218239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.219710 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.219715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.219905 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.220645 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:00.220678 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:00.225456 (+ 4778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":344,"threads_started":1}
W20260928 15:58:00.226800 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:00.209244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:37321 (local address 127.22.125.254:38121)
0928 15:58:00.209909 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:00.209922 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:00.210211 (+ 289us) server_negotiation.cc:407] Connection header received
0928 15:58:00.212284 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:00.212297 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:00.212787 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:00.213259 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:00.214932 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.216895 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.218373 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.219502 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.219690 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.225784 (+ 6094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:37321: BlockingRecv error: recv got EOF from 127.22.125.193:37321 (error 108)
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":255,"threads_started":1}
W20260928 15:58:00.227471 32075 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:38121 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.125.254:38121: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:00.237265 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:00.228109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:56781)
0928 15:58:00.228379 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:00.228401 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:00.228514 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:00.228813 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:00.228820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:00.229265 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:00.229468 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.229481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.230478 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.230486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.232216 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.232236 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.232879 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.232888 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.233114 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.233702 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:00.233732 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:00.236980 (+ 3248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135}
W20260928 15:58:00.237385 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:00.228256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:56781 (local address 127.22.125.254:38121)
0928 15:58:00.228414 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:00.228418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:00.228452 (+ 34us) server_negotiation.cc:407] Connection header received
0928 15:58:00.228620 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:00.228623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:00.228678 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:00.228756 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:00.229611 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.230334 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.232359 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.232731 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.232821 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.237171 (+ 4350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:56781: BlockingRecv error: recv got EOF from 127.22.125.193:56781 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:00.245730 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:00.238173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:45443)
0928 15:58:00.238331 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:00.238356 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:00.238431 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:00.238866 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:00.238871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:00.239150 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:00.239283 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.239291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.240306 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.240312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.241199 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.241209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.241986 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.241991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.242187 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.242746 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:00.242764 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:00.245446 (+ 2682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 15:58:00.245878 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:00.238412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:45443 (local address 127.22.125.254:38121)
0928 15:58:00.238590 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:00.238594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:00.238607 (+ 13us) server_negotiation.cc:407] Connection header received
0928 15:58:00.238643 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:00.238646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:00.238709 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:00.238813 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:00.239420 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.240174 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.241334 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.241839 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.241964 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.245625 (+ 3661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:45443: BlockingRecv error: recv got EOF from 127.22.125.193:45443 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260928 15:58:00.246253 32075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 15:58:00.538812 32079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:00.539072 32079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:00.539167 32079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:00.546289 32079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:00.546401 32079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:00.546523 32079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:58:00.556384 32079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.194.keytab
--principal=oryx/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:42743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=45991
--tserver_master_addrs=127.22.125.254:38121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:00.557653 32079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:00.558966 32079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:00.571547 32087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:00.571635 32084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:00.571616 32085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:00.572829 32079 server_base.cc:1034] running on GCE node
I20260928 15:58:00.573338 32079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:00.574179 32079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:00.575376 32079 hybrid_clock.cc:648] HybridClock initialized: now 1790611080575319 us; error 53 us; skew 500 ppm
I20260928 15:58:00.575641 32079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611080, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:58:00.585439 32079 init.cc:377] Logged in from keytab as oryx/127.22.125.194@KRBTEST.COM (short username oryx)
I20260928 15:58:00.586905 32079 webserver.cc:460] Webserver started at http://127.22.125.194:45991/ using document root <none> and password file <none>
I20260928 15:58:00.587546 32079 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:00.587658 32079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:00.592839 32079 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:58:00.595474 32094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:00.596829 32079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:58:00.597018 32079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "f10d5bc8d5d14527ae31dfcc28ce6511"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "5deffd2f43cfe2debf2e4989eca91bc5"
server_key_iv: "1ad7b800292ec1efbe4e7bff3a2c93b2"
server_key_version: "encryptionkey@0"
I20260928 15:58:00.597954 32079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:00.611954 32079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:00.615940 32079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:00.616235 32079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:00.617157 32079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:00.618425 32079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:00.618552 32079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:00.618683 32079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:00.618772 32079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:00.651270 32079 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:42743
I20260928 15:58:00.651357 32206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:42743 every 8 connection(s)
I20260928 15:58:00.652834 32079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:58:00.658600 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 32079
I20260928 15:58:00.659255 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:42629
--local_ip_for_outbound_sockets=127.22.125.195
--tserver_master_addrs=127.22.125.254:38121
--webserver_port=42231
--webserver_interface=127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 15:58:00.670063 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:00.655475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:54059 (local address 127.22.125.254:38121)
0928 15:58:00.655682 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:00.655688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:00.656847 (+ 1159us) server_negotiation.cc:407] Connection header received
0928 15:58:00.658636 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:00.658642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:00.658732 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:00.658852 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:00.660452 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.661613 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.663111 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.663863 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.664003 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.669877 (+ 5874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:54059: BlockingRecv error: recv got EOF from 127.22.125.194:54059 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 15:58:00.671152 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:00.655968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:54059)
0928 15:58:00.656692 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:00.656731 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:00.658364 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:00.659059 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:00.659067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:00.659653 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:00.660277 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.660295 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.661759 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.661763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.662962 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.662974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.664102 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.664110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.664424 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.665185 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:00.665217 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:00.669735 (+ 4518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":258,"threads_started":1}
W20260928 15:58:00.671804 32207 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:38121 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.125.254:38121: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 15:58:00.680945 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:00.672538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:60991 (local address 127.22.125.254:38121)
0928 15:58:00.672702 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:00.672707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:00.672723 (+ 16us) server_negotiation.cc:407] Connection header received
0928 15:58:00.672821 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:00.672827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:00.672899 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:00.672998 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:00.673608 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.674347 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.675508 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.675905 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.676086 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.680762 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:60991: BlockingRecv error: recv got EOF from 127.22.125.194:60991 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 15:58:00.680941 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:00.672430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:60991)
0928 15:58:00.672603 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:00.672622 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:00.672720 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:00.673021 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:00.673025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:00.673350 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:00.673473 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.673480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.674478 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.674484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.675364 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.675374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.676124 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.676135 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.676380 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.676993 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:00.677023 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:00.680597 (+ 3574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:00 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:00.688999 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:00.681600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:41015)
0928 15:58:00.681855 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:00.681874 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:00.681977 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:00.682276 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:00.682281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:00.682574 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:00.682709 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.682717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.683651 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.683656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.684606 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:00.684617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.685281 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.685287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:00.685493 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.686046 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:00.686062 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:00.688700 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128}
W20260928 15:58:00.689108 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:00.681721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:41015 (local address 127.22.125.254:38121)
0928 15:58:00.681892 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:00.681899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:00.681916 (+ 17us) server_negotiation.cc:407] Connection header received
0928 15:58:00.682102 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:00.682106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:00.682159 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:00.682256 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:00.682842 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.683519 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.684752 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:00.685142 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:00.685279 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:00.688888 (+ 3609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:41015: BlockingRecv error: recv got EOF from 127.22.125.194:41015 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260928 15:58:00.689303 32207 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 15:58:01.000507 32210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:01.000826 32210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:01.000991 32210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:01.008499 32210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:01.008651 32210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:01.008816 32210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:58:01.018579 32210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/krb5kdc/oryx_127.22.125.195.keytab
--principal=oryx/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:42629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=42231
--tserver_master_addrs=127.22.125.254:38121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:01.019850 32210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:01.021191 32210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:01.034281 32215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:01.034200 32218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:01.034250 32216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:01.035120 32210 server_base.cc:1034] running on GCE node
I20260928 15:58:01.035755 32210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:01.036680 32210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:01.037889 32210 hybrid_clock.cc:648] HybridClock initialized: now 1790611081037822 us; error 61 us; skew 500 ppm
I20260928 15:58:01.038151 32210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611081, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:58:01.041216 32210 init.cc:377] Logged in from keytab as oryx/127.22.125.195@KRBTEST.COM (short username oryx)
I20260928 15:58:01.042520 32210 webserver.cc:460] Webserver started at http://127.22.125.195:42231/ using document root <none> and password file <none>
I20260928 15:58:01.043138 32210 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:01.043246 32210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:01.048353 32210 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 15:58:01.050889 32225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:01.052259 32210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:58:01.052436 32210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "42136caf4bb743fabf91fc3f5475a648"
format_stamp: "Formatted at 2026-09-28 15:57:58 on dist-test-slave-9cqd"
server_key: "3aab1054bdc3669ebb6b84f919e0b996"
server_key_iv: "9153cddbb927ccd480beca01bce87158"
server_key_version: "encryptionkey@0"
I20260928 15:58:01.053399 32210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:01.085963 32210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:01.090583 32210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:01.090864 32210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:01.091869 32210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:01.093298 32210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:01.093392 32210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:01.093559 32210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:01.093644 32210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:01.126619 32210 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:42629
I20260928 15:58:01.126708 32337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:42629 every 8 connection(s)
I20260928 15:58:01.128032 32210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:58:01.133937 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 32210
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 15:58:01.144622 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:01.130432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:43299 (local address 127.22.125.254:38121)
0928 15:58:01.130611 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:01.130615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:01.131748 (+ 1133us) server_negotiation.cc:407] Connection header received
0928 15:58:01.133617 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:01.133623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:01.133713 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:01.133824 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:01.135619 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.136520 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.138437 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.138871 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.138959 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.144311 (+ 5352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:43299: BlockingRecv error: recv got EOF from 127.22.125.195:43299 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 15:58:01.145136 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:01.130925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:43299)
0928 15:58:01.131583 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:01.131625 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:01.133346 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:01.134083 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:01.134095 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:01.134633 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:01.135439 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.135465 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.136687 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.136692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.138297 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.138309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.139086 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.139093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.139378 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.140145 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:01.140166 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:01.144027 (+ 3861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":259,"threads_started":1}
W20260928 15:58:01.145821 32338 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:38121 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.125.254:38121: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
I20260928 15:58:01.149992 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:01.134941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.22.125.254:38121)
0928 15:58:01.135631 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:01.135636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:01.135650 (+ 14us) server_negotiation.cc:407] Connection header received
0928 15:58:01.135699 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:01.135703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:01.135772 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:01.135885 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:01.137646 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.138348 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.139972 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.140448 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.140540 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.143204 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:01.143211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:01.143226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:01.143260 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:01.146749 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:01.147383 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:01.147391 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:01.147399 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:01.147558 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:01.147957 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:01.147962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:01.147965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:01.148449 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:01.148765 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:01.149171 (+ 406us) server_negotiation.cc:299] Negotiation successful
0928 15:58:01.149540 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":452,"threads_started":1}
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:01.154189 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:01.146336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:38749)
0928 15:58:01.146554 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:01.146573 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:01.146667 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:01.146962 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:01.146966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:01.147283 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:01.147418 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.147425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.148492 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.148497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.149413 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.149429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.150426 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.150432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.150601 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.151082 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:01.151098 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:01.153957 (+ 2859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260928 15:58:01.154378 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:01.146466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:38749 (local address 127.22.125.254:38121)
0928 15:58:01.146593 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:01.146597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:01.146614 (+ 17us) server_negotiation.cc:407] Connection header received
0928 15:58:01.146779 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:01.146783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:01.146830 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:01.146936 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:01.147553 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.148362 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.149562 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.150263 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.150415 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.154112 (+ 3697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:38749: BlockingRecv error: recv got EOF from 127.22.125.195:38749 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:01.162204 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:01.155034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:43809)
0928 15:58:01.155209 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:01.155227 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:01.155315 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:01.155547 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:01.155550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:01.155826 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:01.155968 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.155976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.156928 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.156932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.158217 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.158229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.158943 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.158947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.159078 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.159620 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:01.159636 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:01.162022 (+ 2386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 15:58:01.162384 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:01.155119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:43809 (local address 127.22.125.254:38121)
0928 15:58:01.155313 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:01.155317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:01.155331 (+ 14us) server_negotiation.cc:407] Connection header received
0928 15:58:01.155376 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:01.155379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:01.155434 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:01.155507 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:01.156127 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.156791 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.158336 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.158723 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.158809 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.162177 (+ 3368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:43809: BlockingRecv error: recv got EOF from 127.22.125.195:43809 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260928 15:58:01.162531 32338 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:01.255659 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:01.246854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:58549)
0928 15:58:01.247411 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:01.247430 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:01.247579 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:01.247922 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:01.247926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:01.248443 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:01.248631 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.248640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.249663 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.249668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.250633 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.250645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.251419 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.251425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.251589 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.252185 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:01.252203 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:01.255309 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":288,"threads_started":1}
W20260928 15:58:01.255770 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:01.246905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:58549 (local address 127.22.125.254:38121)
0928 15:58:01.247132 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:01.247139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:01.247508 (+ 369us) server_negotiation.cc:407] Connection header received
0928 15:58:01.247706 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:01.247711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:01.247783 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:01.247898 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:01.248785 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.249498 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.250791 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.251231 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.251366 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.255542 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:58549: BlockingRecv error: recv got EOF from 127.22.125.193:58549 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:01 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:01.698283 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:01.689860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:34907)
0928 15:58:01.690371 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:01.690386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:01.690539 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:01.690876 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:01.690880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:01.691361 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:01.691549 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.691558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.692630 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.692636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.693594 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:01.693604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.694472 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.694478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:01.694647 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.695327 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:01.695345 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:01.698062 (+ 2717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":278,"threads_started":1}
W20260928 15:58:01.698519 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:01.689952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:34907 (local address 127.22.125.254:38121)
0928 15:58:01.690180 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:01.690190 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:01.690476 (+ 286us) server_negotiation.cc:407] Connection header received
0928 15:58:01.690665 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:01.690676 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:01.690737 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:01.690849 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:01.691700 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.692488 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.693740 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:01.694292 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:01.694437 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:01.698253 (+ 3816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:34907: BlockingRecv error: recv got EOF from 127.22.125.194:34907 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 28 15:58:02 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:02 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:02.172303 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:02.163108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:54805)
0928 15:58:02.163769 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:02.163785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:02.163936 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:02.164474 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:02.164480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:02.165078 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:02.165349 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:02.165366 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.166379 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.166383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:02.167319 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:02.167329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.168066 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.168071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:02.168187 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:02.168809 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:02.168826 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:02.172070 (+ 3244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":265,"threads_started":1}
W20260928 15:58:02.172492 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:02.163159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:54805 (local address 127.22.125.254:38121)
0928 15:58:02.163385 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:02.163394 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:02.163871 (+ 477us) server_negotiation.cc:407] Connection header received
0928 15:58:02.164121 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:02.164127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:02.164247 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:02.164400 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:02.165495 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.166253 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.167465 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.167890 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.167984 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:02.172277 (+ 4293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:54805: BlockingRecv error: recv got EOF from 127.22.125.195:54805 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 28 15:58:02 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:02 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:02.264935 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:02.256604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:54799)
0928 15:58:02.257182 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:02.257204 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:02.257377 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:02.257728 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:02.257732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:02.258246 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:02.258428 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:02.258437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.259449 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.259456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:02.260409 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:02.260421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.261150 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.261155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:02.261322 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:02.261865 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:02.261882 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:02.264687 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":320,"threads_started":1}
W20260928 15:58:02.265118 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:02.256696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:54799 (local address 127.22.125.254:38121)
0928 15:58:02.256915 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:02.256923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:02.257302 (+ 379us) server_negotiation.cc:407] Connection header received
0928 15:58:02.257501 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:02.257508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:02.257597 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:02.257701 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:02.258575 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.259319 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.260551 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.260988 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.261121 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:02.264873 (+ 3752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:54799: BlockingRecv error: recv got EOF from 127.22.125.193:54799 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 28 15:58:02 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:02 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:02.707569 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:02.699376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:46899)
0928 15:58:02.699846 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:02.699874 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:02.699986 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:02.700391 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:02.700394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:02.700769 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:02.700960 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:02.700969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.702031 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.702035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:02.703117 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:02.703129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.703812 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.703816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:02.703935 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:02.704558 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:02.704575 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:02.707327 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":259,"threads_started":1}
W20260928 15:58:02.707724 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:02.699517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:46899 (local address 127.22.125.254:38121)
0928 15:58:02.699751 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:02.699757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:02.699952 (+ 195us) server_negotiation.cc:407] Connection header received
0928 15:58:02.700200 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:02.700204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:02.700263 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:02.700359 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:02.701097 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.701903 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.703242 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:02.703688 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:02.703783 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:02.707517 (+ 3734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:46899: BlockingRecv error: recv got EOF from 127.22.125.194:46899 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 28 15:58:03 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:03 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:03.181396 32348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:03.173226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:50641)
0928 15:58:03.173791 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:03.173807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:03.173916 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:03.174203 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:03.174206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:03.174596 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:03.174765 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:03.174773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.175733 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.175736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:03.176630 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:03.176641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.177277 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.177281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:03.177407 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:03.178066 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:03.178094 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:03.181151 (+ 3057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":321,"threads_started":1}
W20260928 15:58:03.181560 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:03.173424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:50641 (local address 127.22.125.254:38121)
0928 15:58:03.173613 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:03.173617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:03.173882 (+ 265us) server_negotiation.cc:407] Connection header received
0928 15:58:03.174012 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:03.174015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:03.174075 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:03.174158 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:03.174885 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.175621 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.176748 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.177154 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.177245 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:03.181328 (+ 4083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:50641: BlockingRecv error: recv got EOF from 127.22.125.195:50641 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 28 15:58:03 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:03 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:03.274461 32349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:03.265953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:52657)
0928 15:58:03.266446 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:03.266462 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:03.266621 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:03.266969 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:03.266975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:03.267345 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:03.267521 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:03.267531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.268658 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.268667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:03.269574 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:03.269586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.270250 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.270255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:03.270442 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:03.271085 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:03.271103 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:03.274208 (+ 3105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":272,"threads_started":1}
W20260928 15:58:03.274648 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:03.266003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:52657 (local address 127.22.125.254:38121)
0928 15:58:03.266242 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:03.266251 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:03.266553 (+ 302us) server_negotiation.cc:407] Connection header received
0928 15:58:03.266747 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:03.266759 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:03.266820 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:03.266934 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:03.267669 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.268518 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.269713 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.270103 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.270237 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:03.274402 (+ 4165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:52657: BlockingRecv error: recv got EOF from 127.22.125.193:52657 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 28 15:58:03 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:03 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:03.717021 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:03.708610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:46871)
0928 15:58:03.709122 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:03.709143 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:03.709282 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:03.709646 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:03.709654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:03.710030 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:03.710203 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:03.710212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.711258 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.711266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:03.712202 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:03.712213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.712973 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.712979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:03.713172 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:03.713768 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:03.713810 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:03.716769 (+ 2959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":269,"threads_started":1}
W20260928 15:58:03.717190 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:03.708885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:46871 (local address 127.22.125.254:38121)
0928 15:58:03.709103 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:03.709110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:03.709206 (+ 96us) server_negotiation.cc:407] Connection header received
0928 15:58:03.709430 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:03.709435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:03.709503 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:03.709606 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:03.710361 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.711128 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.712349 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:03.712823 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:03.712951 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:03.716955 (+ 4004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:46871: BlockingRecv error: recv got EOF from 127.22.125.194:46871 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 28 15:58:04 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:04 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:04.190256 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:04.182273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:38877)
0928 15:58:04.182787 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:04.182803 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:04.182925 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:04.183217 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:04.183220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:04.183559 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:04.183724 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:04.183732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.184697 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.184701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:04.185595 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:04.185606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.186218 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.186222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:04.186349 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:04.186982 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:04.187000 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:04.189865 (+ 2865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":292,"threads_started":1}
W20260928 15:58:04.190286 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:04.182389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:38877 (local address 127.22.125.254:38121)
0928 15:58:04.182549 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:04.182553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:04.182874 (+ 321us) server_negotiation.cc:407] Connection header received
0928 15:58:04.183018 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:04.183022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:04.183094 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:04.183174 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:04.183836 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.184586 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.185711 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.186090 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.186188 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:04.190072 (+ 3884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:38877: BlockingRecv error: recv got EOF from 127.22.125.195:38877 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 28 15:58:04 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:04 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:04.283532 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:04.275315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:42231)
0928 15:58:04.275814 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:04.275831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:04.276018 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:04.276395 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:04.276399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:04.276804 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:04.276974 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:04.276983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.277986 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.277991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:04.278963 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:04.278975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.279681 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.279688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:04.279875 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:04.280556 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:04.280575 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:04.283301 (+ 2726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":268,"threads_started":1}
W20260928 15:58:04.283784 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:04.275404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:42231 (local address 127.22.125.254:38121)
0928 15:58:04.275642 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:04.275649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:04.275931 (+ 282us) server_negotiation.cc:407] Connection header received
0928 15:58:04.276187 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:04.276196 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:04.276257 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:04.276369 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:04.277117 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.277846 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.279114 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.279530 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.279670 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:04.283515 (+ 3845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:42231: BlockingRecv error: recv got EOF from 127.22.125.193:42231 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 28 15:58:04 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:04 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:04.726215 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:04.717982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:37715)
0928 15:58:04.718529 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:04.718545 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:04.718657 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:04.718934 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:04.718937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:04.719292 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:04.719461 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:04.719469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.720490 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.720493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:04.721454 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:04.721465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.722095 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.722099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:04.722216 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:04.722840 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:04.722859 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:04.725979 (+ 3120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":320,"threads_started":1}
W20260928 15:58:04.726354 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:04.718115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:37715 (local address 127.22.125.254:38121)
0928 15:58:04.718294 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:04.718298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:04.718617 (+ 319us) server_negotiation.cc:407] Connection header received
0928 15:58:04.718741 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:04.718744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:04.718804 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:04.718889 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:04.719581 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.720372 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.721573 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:04.721982 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:04.722068 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:04.726148 (+ 4080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:37715: BlockingRecv error: recv got EOF from 127.22.125.194:37715 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 28 15:58:05 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:05 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:05.199193 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:05.191211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:51447)
0928 15:58:05.191727 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:05.191743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:05.191852 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:05.192199 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:05.192202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:05.192562 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:05.192753 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:05.192774 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.193765 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.193768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:05.194728 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:05.194739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.195381 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.195384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:05.195494 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:05.196127 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:05.196144 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:05.198849 (+ 2705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":291,"threads_started":1}
W20260928 15:58:05.199317 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:05.191257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:51447 (local address 127.22.125.254:38121)
0928 15:58:05.191413 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:05.191417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:05.191813 (+ 396us) server_negotiation.cc:407] Connection header received
0928 15:58:05.191966 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:05.191970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:05.192028 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:05.192165 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:05.192891 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.193653 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.194871 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.195268 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.195351 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:05.199062 (+ 3711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:51447: BlockingRecv error: recv got EOF from 127.22.125.195:51447 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 15:58:05 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:05 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:05.292294 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:05.284458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:41011)
0928 15:58:05.285018 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:05.285035 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:05.285144 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:05.285433 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:05.285436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:05.285841 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:05.285998 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:05.286007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.287029 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.287032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:05.287942 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:05.287955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.288627 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.288631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:05.288759 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:05.289334 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:05.289351 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:05.291990 (+ 2639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":292,"threads_started":1}
W20260928 15:58:05.292497 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:05.284560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:41011 (local address 127.22.125.254:38121)
0928 15:58:05.284738 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:05.284742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:05.285110 (+ 368us) server_negotiation.cc:407] Connection header received
0928 15:58:05.285240 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:05.285244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:05.285303 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:05.285387 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:05.286144 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.286862 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.288106 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.288508 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.288598 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:05.292250 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:41011: BlockingRecv error: recv got EOF from 127.22.125.193:41011 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 28 15:58:05 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:05 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:05.743625 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:05.727165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:41279)
0928 15:58:05.727718 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:05.727734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:05.727837 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:05.728126 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:05.728130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:05.728592 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:05.728803 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:05.728819 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.730254 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.730260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:05.731171 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:05.731181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.731849 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.731852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:05.731974 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:05.740148 (+ 8174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:05.740167 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:05.743375 (+ 3208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":294,"threads_started":1}
W20260928 15:58:05.743822 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:05.727315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:41279 (local address 127.22.125.254:38121)
0928 15:58:05.727522 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:05.727527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:05.727805 (+ 278us) server_negotiation.cc:407] Connection header received
0928 15:58:05.727934 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:05.727937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:05.727994 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:05.728094 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:05.728993 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.730087 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.731290 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:05.731719 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:05.731809 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:05.743556 (+ 11747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:41279: BlockingRecv error: recv got EOF from 127.22.125.194:41279 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 28 15:58:06 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:06 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:06.209132 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:06.200471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:50291)
0928 15:58:06.201070 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:06.201086 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:06.201270 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:06.201616 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:06.201619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:06.202022 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:06.202231 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:06.202239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.203357 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.203361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:06.204333 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:06.204350 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.205225 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.205229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:06.205361 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:06.205978 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:06.205994 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:06.208886 (+ 2892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":280,"threads_started":1}
W20260928 15:58:06.209304 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:06.200640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:50291 (local address 127.22.125.254:38121)
0928 15:58:06.200860 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:06.200867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:06.201171 (+ 304us) server_negotiation.cc:407] Connection header received
0928 15:58:06.201415 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:06.201420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:06.201495 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:06.201574 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:06.202414 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.203229 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.204457 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.205095 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.205222 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:06.209084 (+ 3862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:50291: BlockingRecv error: recv got EOF from 127.22.125.195:50291 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 28 15:58:06 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:06 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:06.301506 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:06.293165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:53221)
0928 15:58:06.293662 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:06.293677 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:06.293790 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:06.294102 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:06.294106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:06.294468 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:06.294617 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:06.294625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.296141 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.296146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:06.297126 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:06.297137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.297765 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.297769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:06.297892 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:06.298472 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:06.298488 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:06.301271 (+ 2783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":298,"threads_started":1}
W20260928 15:58:06.301736 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:06.293276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:53221 (local address 127.22.125.254:38121)
0928 15:58:06.293532 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:06.293539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:06.293758 (+ 219us) server_negotiation.cc:407] Connection header received
0928 15:58:06.293897 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:06.293901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:06.293960 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:06.294062 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:06.294752 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.295934 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.297264 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.297639 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.297728 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:06.301445 (+ 3717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:53221: BlockingRecv error: recv got EOF from 127.22.125.193:53221 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Sep 28 15:58:06 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:06 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:06.757462 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:06.744558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:42781)
0928 15:58:06.747410 (+ 2852us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:06.747433 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:06.747590 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:06.747918 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:06.747926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:06.748499 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:06.748722 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:06.748735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.749736 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.749743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:06.751449 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:06.751467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.752541 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.752548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:06.752766 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:06.753484 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:06.753512 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:06.757182 (+ 3670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2685,"thread_start_us":2534,"threads_started":1}
W20260928 15:58:06.757849 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:06.744631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:42781 (local address 127.22.125.254:38121)
0928 15:58:06.744885 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:06.744896 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:06.747531 (+ 2635us) server_negotiation.cc:407] Connection header received
0928 15:58:06.747702 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:06.747705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:06.747767 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:06.747848 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:06.748867 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.749586 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.751663 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:06.752370 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:06.752465 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:06.757533 (+ 5068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:42781: BlockingRecv error: recv got EOF from 127.22.125.194:42781 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 28 15:58:07 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:07 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:07.218873 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:07.210086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:59749)
0928 15:58:07.210622 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:07.210638 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:07.210747 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:07.211044 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:07.211047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:07.211407 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:07.211564 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:07.211572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.212835 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.212841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:07.214396 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:07.214409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.215087 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.215090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:07.215211 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:07.215759 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:07.215776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:07.218558 (+ 2782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":302,"threads_started":1}
W20260928 15:58:07.219002 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:07.210286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:59749 (local address 127.22.125.254:38121)
0928 15:58:07.210474 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:07.210478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:07.210713 (+ 235us) server_negotiation.cc:407] Connection header received
0928 15:58:07.210855 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:07.210858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:07.210919 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:07.211001 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:07.211684 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.212670 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.214507 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.214948 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.215052 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:07.218746 (+ 3694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:59749: BlockingRecv error: recv got EOF from 127.22.125.195:59749 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 28 15:58:07 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:07 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:07.311060 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:07.302459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:39023)
0928 15:58:07.303133 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:07.303150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:07.303270 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:07.303576 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:07.303580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:07.304198 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:07.304371 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:07.304380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.305324 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.305328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:07.306377 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:07.306389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.307143 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.307148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:07.307337 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:07.307931 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:07.307958 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:07.310804 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":355,"threads_started":1}
W20260928 15:58:07.311187 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:07.302487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:39023 (local address 127.22.125.254:38121)
0928 15:58:07.302745 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:07.302755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:07.303233 (+ 478us) server_negotiation.cc:407] Connection header received
0928 15:58:07.303354 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:07.303357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:07.303417 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:07.303498 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:07.304508 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.305197 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.306523 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.306968 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.307104 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:07.310997 (+ 3893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:39023: BlockingRecv error: recv got EOF from 127.22.125.193:39023 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 28 15:58:07 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:07 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:07.767804 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:07.758786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:50147)
0928 15:58:07.759354 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:07.759376 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:07.759512 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:07.759837 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:07.759842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:07.760412 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:07.760625 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:07.760639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.761677 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.761682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:07.762932 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:07.762946 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.763629 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.763633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:07.763756 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:07.764362 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:07.764381 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:07.767537 (+ 3156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":253,"threads_started":1}
W20260928 15:58:07.767942 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:07.758850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:50147 (local address 127.22.125.254:38121)
0928 15:58:07.759090 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:07.759100 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:07.759463 (+ 363us) server_negotiation.cc:407] Connection header received
0928 15:58:07.759611 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:07.759615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:07.759698 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:07.759777 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:07.760762 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.761543 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.763058 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:07.763483 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:07.763575 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:07.767705 (+ 4130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:50147: BlockingRecv error: recv got EOF from 127.22.125.194:50147 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 28 15:58:08 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:08 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 15:58:08.229202 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:08.219965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:37991 (local address 127.22.125.254:38121)
0928 15:58:08.220148 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:08.220153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:08.220607 (+ 454us) server_negotiation.cc:407] Connection header received
0928 15:58:08.220808 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:08.220813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:08.220885 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:08.220986 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:08.221792 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.222520 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.223738 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.224244 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.224385 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:08.229032 (+ 4647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:37991: BlockingRecv error: recv got EOF from 127.22.125.195:37991 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 15:58:08.229279 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:08.219923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:37991)
0928 15:58:08.220508 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:08.220523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:08.220672 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:08.221022 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:08.221027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:08.221483 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:08.221646 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:08.221656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.222663 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.222669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:08.223593 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:08.223604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.224392 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.224400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:08.224572 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:08.225270 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:08.225287 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:08.228866 (+ 3579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":277,"threads_started":1}
Sep 28 15:58:08 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:08 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:08.322810 32364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:08.312202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:46373)
0928 15:58:08.313006 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:08.313031 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:08.313250 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:08.313640 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:08.313644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:08.314203 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:08.314447 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:08.314464 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.315502 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.315510 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:08.317122 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:08.317161 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.317951 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.317960 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:08.318190 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:08.318895 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:08.318925 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:08.322468 (+ 3543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":391,"threads_started":1}
W20260928 15:58:08.322930 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:08.312682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:46373 (local address 127.22.125.254:38121)
0928 15:58:08.312952 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:08.312961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:08.313131 (+ 170us) server_negotiation.cc:407] Connection header received
0928 15:58:08.313390 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:08.313398 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:08.313499 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:08.313604 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:08.314628 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.315354 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.317287 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.317750 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.317845 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:08.322672 (+ 4827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:46373: BlockingRecv error: recv got EOF from 127.22.125.193:46373 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Sep 28 15:58:08 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:08 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:08.777374 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:08.768820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:36615)
0928 15:58:08.769334 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:08.769350 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:08.769520 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:08.769861 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:08.769865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:08.770278 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:08.770429 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:08.770437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.771609 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.771614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:08.772870 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:08.772885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.773783 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.773787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:08.773908 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:08.774465 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:08.774482 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:08.777133 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":240,"threads_started":1}
W20260928 15:58:08.777530 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:08.768899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:36615 (local address 127.22.125.254:38121)
0928 15:58:08.769118 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:08.769127 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:08.769445 (+ 318us) server_negotiation.cc:407] Connection header received
0928 15:58:08.769642 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:08.769649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:08.769716 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:08.769837 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:08.770570 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.771463 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.773049 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:08.773582 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:08.773716 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:08.777333 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:36615: BlockingRecv error: recv got EOF from 127.22.125.194:36615 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 28 15:58:09 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:09 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:09.238412 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:09.230228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:40765)
0928 15:58:09.230778 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:09.230793 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:09.230963 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:09.231291 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:09.231296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:09.231722 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:09.231877 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:09.231886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.232956 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.232962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:09.233894 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:09.233905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.234642 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.234648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:09.234822 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:09.235473 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:09.235490 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:09.238187 (+ 2697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":304,"threads_started":1}
W20260928 15:58:09.238605 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:09.230316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:40765 (local address 127.22.125.254:38121)
0928 15:58:09.230556 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:09.230565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:09.230897 (+ 332us) server_negotiation.cc:407] Connection header received
0928 15:58:09.231087 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:09.231098 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:09.231157 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:09.231259 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:09.232073 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.232790 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.234042 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.234460 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.234595 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:09.238376 (+ 3781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:40765: BlockingRecv error: recv got EOF from 127.22.125.195:40765 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 28 15:58:09 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:09 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:09.332350 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:09.323910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:56707)
0928 15:58:09.324506 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:09.324522 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:09.324634 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:09.324935 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:09.324939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:09.325426 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:09.325596 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:09.325605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.326694 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.326697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:09.327606 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:09.327617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.328531 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.328536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:09.328706 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:09.329294 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:09.329312 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:09.332111 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":344,"threads_started":1}
W20260928 15:58:09.332541 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:09.323958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:56707 (local address 127.22.125.254:38121)
0928 15:58:09.324170 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:09.324175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:09.324590 (+ 415us) server_negotiation.cc:407] Connection header received
0928 15:58:09.324735 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:09.324738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:09.324799 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:09.324886 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:09.325715 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.326584 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.327738 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.328370 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.328500 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:09.332294 (+ 3794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:56707: BlockingRecv error: recv got EOF from 127.22.125.193:56707 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 15:58:09 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:09 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:09.786337 32368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:09.778422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:37399)
0928 15:58:09.778954 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:09.778970 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:09.779082 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:09.779397 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:09.779401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:09.779799 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:09.779979 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:09.779987 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.781022 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.781025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:09.781956 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:09.781968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.782593 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.782597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:09.782716 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:09.783375 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:09.783392 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:09.786069 (+ 2677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":287,"threads_started":1}
W20260928 15:58:09.786466 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:09.778472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:37399 (local address 127.22.125.254:38121)
0928 15:58:09.778634 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:09.778638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:09.779043 (+ 405us) server_negotiation.cc:407] Connection header received
0928 15:58:09.779182 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:09.779185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:09.779244 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:09.779325 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:09.780124 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.780912 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.782072 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:09.782471 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:09.782559 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:09.786257 (+ 3698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:37399: BlockingRecv error: recv got EOF from 127.22.125.194:37399 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 28 15:58:10 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:10 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:10.247653 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:10.239361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:45391)
0928 15:58:10.239882 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:10.239898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:10.240064 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:10.240397 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:10.240404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:10.240816 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:10.240979 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:10.240988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.242008 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.242014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:10.242956 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:10.242968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.243677 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.243682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:10.243854 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:10.244487 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:10.244504 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:10.247316 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":268,"threads_started":1}
W20260928 15:58:10.247783 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:10.239435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:45391 (local address 127.22.125.254:38121)
0928 15:58:10.239669 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:10.239678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:10.239983 (+ 305us) server_negotiation.cc:407] Connection header received
0928 15:58:10.240184 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:10.240189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:10.240256 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:10.240354 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:10.241124 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.241875 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.243101 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.243517 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.243664 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:10.247527 (+ 3863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:45391: BlockingRecv error: recv got EOF from 127.22.125.195:45391 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 28 15:58:10 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:10 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:10.341450 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:10.333245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:49463)
0928 15:58:10.333779 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:10.333796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:10.333907 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:10.334188 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:10.334191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:10.334558 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:10.334702 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:10.334710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.335692 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.335695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:10.336740 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:10.336752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.337432 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.337436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:10.337539 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:10.338094 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:10.338111 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:10.341128 (+ 3017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":320,"threads_started":1}
W20260928 15:58:10.341631 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:10.333345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:49463 (local address 127.22.125.254:38121)
0928 15:58:10.333524 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:10.333528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:10.333877 (+ 349us) server_negotiation.cc:407] Connection header received
0928 15:58:10.334004 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:10.334007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:10.334066 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:10.334145 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:10.334822 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.335576 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.336865 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.337290 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.337388 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:10.341397 (+ 4009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:49463: BlockingRecv error: recv got EOF from 127.22.125.193:49463 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 28 15:58:10 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:10 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:10.807492 32371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:10.787412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:47929)
0928 15:58:10.787927 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:10.787944 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:10.788111 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:10.788373 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:10.788376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:10.788718 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:10.788865 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:10.788874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.789807 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.789810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:10.790735 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:10.790746 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.791398 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.791402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:10.791516 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:10.804239 (+ 12723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:10.804259 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:10.807252 (+ 2993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":284,"threads_started":1}
W20260928 15:58:10.807852 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:10.787486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:47929 (local address 127.22.125.254:38121)
0928 15:58:10.787726 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:10.787733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:10.788026 (+ 293us) server_negotiation.cc:407] Connection header received
0928 15:58:10.788211 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:10.788215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:10.788275 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:10.788353 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:10.788981 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.789689 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.790849 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:10.791267 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:10.791356 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:10.807465 (+ 16109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:47929: BlockingRecv error: recv got EOF from 127.22.125.194:47929 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 28 15:58:11 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:11 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:11.257326 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:11.248713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:36651)
0928 15:58:11.249280 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:11.249296 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:11.249457 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:11.249850 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:11.249855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:11.250246 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:11.250386 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:11.250394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.251455 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.251461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:11.252424 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:11.252435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.253219 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.253223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:11.253419 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:11.254012 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:11.254029 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:11.256961 (+ 2932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":327,"threads_started":1}
W20260928 15:58:11.257416 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:11.248831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:36651 (local address 127.22.125.254:38121)
0928 15:58:11.249025 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:11.249029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:11.249393 (+ 364us) server_negotiation.cc:407] Connection header received
0928 15:58:11.249585 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:11.249591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:11.249703 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:11.249812 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:11.250534 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.251312 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.252573 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.253075 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.253196 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:11.257175 (+ 3979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:36651: BlockingRecv error: recv got EOF from 127.22.125.195:36651 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 28 15:58:11 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:11 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:11.351344 32373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:11.342519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:45289)
0928 15:58:11.343065 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:11.343092 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:11.343249 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:11.343641 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:11.343646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:11.344177 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:11.344355 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:11.344365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.345591 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.345598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:11.346510 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:11.346522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.347264 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.347270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:11.347454 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:11.348128 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:11.348147 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:11.350992 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":317,"threads_started":1}
W20260928 15:58:11.351581 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:11.342599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:45289 (local address 127.22.125.254:38121)
0928 15:58:11.342784 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:11.342788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:11.343185 (+ 397us) server_negotiation.cc:407] Connection header received
0928 15:58:11.343389 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:11.343395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:11.343494 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:11.343602 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:11.344500 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.345454 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.346657 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.347098 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.347232 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:11.351191 (+ 3959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:45289: BlockingRecv error: recv got EOF from 127.22.125.193:45289 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 28 15:58:11 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:11 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:11.816782 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:11.808502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:36511)
0928 15:58:11.809088 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:11.809108 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:11.809242 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:11.809549 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:11.809552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:11.809946 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:11.810119 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:11.810127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.811179 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.811182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:11.812091 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:11.812102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.812770 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.812774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:11.812970 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:11.813635 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:11.813653 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:11.816452 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":312,"threads_started":1}
W20260928 15:58:11.816900 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:11.808601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:36511 (local address 127.22.125.254:38121)
0928 15:58:11.808839 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:11.808846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:11.809195 (+ 349us) server_negotiation.cc:407] Connection header received
0928 15:58:11.809362 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:11.809366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:11.809426 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:11.809505 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:11.810233 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.811039 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.812203 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:11.812645 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:11.812737 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:11.816661 (+ 3924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:36511: BlockingRecv error: recv got EOF from 127.22.125.194:36511 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 28 15:58:12 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:12 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:12.266404 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:12.258149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:44743)
0928 15:58:12.258597 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:12.258613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:12.258743 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:12.258993 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:12.258995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:12.259364 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:12.259546 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:12.259554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.260603 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.260606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:12.261491 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:12.261501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.262092 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.262096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:12.262232 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:12.262906 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:12.262924 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:12.266030 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":255,"threads_started":1}
W20260928 15:58:12.266458 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:12.258252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:44743 (local address 127.22.125.254:38121)
0928 15:58:12.258482 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:12.258488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:12.258711 (+ 223us) server_negotiation.cc:407] Connection header received
0928 15:58:12.258834 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:12.258837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:12.258901 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:12.258979 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:12.259676 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.260489 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.261602 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.261976 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.262060 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:12.266235 (+ 4175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:44743: BlockingRecv error: recv got EOF from 127.22.125.195:44743 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 28 15:58:12 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:12 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:12.360796 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:12.352268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:53835)
0928 15:58:12.352776 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:12.352793 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:12.352961 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:12.353327 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:12.353332 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:12.353769 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:12.353937 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:12.353946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.354950 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.354957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:12.355882 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:12.355893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.356716 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.356722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:12.356977 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:12.357535 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:12.357552 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:12.360512 (+ 2960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":283,"threads_started":1}
W20260928 15:58:12.361012 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:12.352331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:53835 (local address 127.22.125.254:38121)
0928 15:58:12.352548 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:12.352556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:12.352886 (+ 330us) server_negotiation.cc:407] Connection header received
0928 15:58:12.353108 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:12.353112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:12.353171 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:12.353275 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:12.354087 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.354808 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.356026 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.356535 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.356677 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:12.360743 (+ 4066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:53835: BlockingRecv error: recv got EOF from 127.22.125.193:53835 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 15:58:12 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:12 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:12.825848 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:12.817817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:42325)
0928 15:58:12.818338 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:12.818357 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:12.818469 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:12.818782 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:12.818785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:12.819215 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:12.819363 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:12.819371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.820396 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.820399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:12.821310 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:12.821321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.822003 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.822008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:12.822135 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:12.822757 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:12.822775 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:12.825607 (+ 2832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":297,"threads_started":1}
W20260928 15:58:12.826084 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:12.817885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:42325 (local address 127.22.125.254:38121)
0928 15:58:12.818092 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:12.818096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:12.818433 (+ 337us) server_negotiation.cc:407] Connection header received
0928 15:58:12.818591 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:12.818595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:12.818659 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:12.818741 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:12.819495 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.820279 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.821436 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:12.821851 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:12.821946 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:12.825818 (+ 3872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:42325: BlockingRecv error: recv got EOF from 127.22.125.194:42325 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 28 15:58:13 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:13 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:13.275478 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:13.267309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:44819)
0928 15:58:13.267799 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:13.267815 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:13.267967 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:13.268329 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:13.268334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:13.268843 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:13.269062 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:13.269077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.270212 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.270216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:13.271124 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:13.271135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.271757 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.271761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:13.271895 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:13.272531 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:13.272548 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:13.275257 (+ 2709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":277,"threads_started":1}
W20260928 15:58:13.275707 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:13.267372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:44819 (local address 127.22.125.254:38121)
0928 15:58:13.267590 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:13.267600 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:13.267895 (+ 295us) server_negotiation.cc:407] Connection header received
0928 15:58:13.268111 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:13.268116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:13.268183 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:13.268286 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:13.269214 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.270082 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.271236 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.271643 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.271731 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:13.275450 (+ 3719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:44819: BlockingRecv error: recv got EOF from 127.22.125.195:44819 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 15:58:13 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:13 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:13.369755 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:13.361734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:39475)
0928 15:58:13.362479 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:13.362495 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:13.362615 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:13.362922 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:13.362926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:13.363312 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:13.363482 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:13.363490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.364471 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.364475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:13.365384 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:13.365394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.366064 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.366068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:13.366193 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:13.366756 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:13.366773 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:13.369533 (+ 2760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":481,"threads_started":1}
W20260928 15:58:13.369932 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:13.361776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:39475 (local address 127.22.125.254:38121)
0928 15:58:13.361963 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:13.361969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:13.362574 (+ 605us) server_negotiation.cc:407] Connection header received
0928 15:58:13.362711 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:13.362715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:13.362793 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:13.362875 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:13.363598 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.364352 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.365497 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.365916 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.366005 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:13.369711 (+ 3706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:39475: BlockingRecv error: recv got EOF from 127.22.125.193:39475 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 28 15:58:13 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:13 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:13.835304 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:13.826721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:35795)
0928 15:58:13.827289 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:13.827303 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:13.827461 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:13.827801 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:13.827806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:13.828260 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:13.828430 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:13.828440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.829463 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.829469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:13.830386 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:13.830398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.831111 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.831116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:13.831310 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:13.831872 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:13.831890 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:13.834947 (+ 3057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":328,"threads_started":1}
W20260928 15:58:13.835371 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:13.826922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:35795 (local address 127.22.125.254:38121)
0928 15:58:13.827145 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:13.827152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:13.827387 (+ 235us) server_negotiation.cc:407] Connection header received
0928 15:58:13.827585 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:13.827590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:13.827665 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:13.827765 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:13.828577 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.829317 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.830529 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:13.830968 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:13.831100 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:13.835149 (+ 4049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:35795: BlockingRecv error: recv got EOF from 127.22.125.194:35795 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 28 15:58:14 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:14 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:14.284782 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:14.276339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:51289)
0928 15:58:14.276902 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:14.276918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:14.277028 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:14.277304 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:14.277307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:14.277701 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:14.277859 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:14.277887 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.278833 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.278836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:14.280223 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:14.280242 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.281032 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.281037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:14.281149 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:14.281786 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:14.281804 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:14.284556 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":307,"threads_started":1}
W20260928 15:58:14.284945 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:14.276449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:51289 (local address 127.22.125.254:38121)
0928 15:58:14.276631 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:14.276635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:14.276989 (+ 354us) server_negotiation.cc:407] Connection header received
0928 15:58:14.277123 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:14.277126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:14.277184 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:14.277262 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:14.278014 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.278719 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.280402 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.280877 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.280970 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:14.284742 (+ 3772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:51289: BlockingRecv error: recv got EOF from 127.22.125.195:51289 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 28 15:58:14 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:14 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:14.379192 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:14.370700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:59329)
0928 15:58:14.371211 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:14.371228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:14.371340 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:14.371642 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:14.371645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:14.371995 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:14.372258 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:14.372268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.373345 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.373350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:14.374258 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:14.374270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.375006 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.375012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:14.375190 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:14.375858 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:14.375877 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:14.378862 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":295,"threads_started":1}
W20260928 15:58:14.379331 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:14.370788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:59329 (local address 127.22.125.254:38121)
0928 15:58:14.371024 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:14.371029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:14.371297 (+ 268us) server_negotiation.cc:407] Connection header received
0928 15:58:14.371443 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:14.371446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:14.371517 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:14.371596 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:14.372416 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.373195 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.374408 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.374845 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.374984 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:14.379088 (+ 4104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:59329: BlockingRecv error: recv got EOF from 127.22.125.193:59329 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Sep 28 15:58:14 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:14 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:14.844555 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:14.836376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:53917)
0928 15:58:14.836819 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:14.836835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:14.836949 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:14.837222 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:14.837225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:14.837567 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:14.837746 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:14.837756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.838704 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.838707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:14.839615 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:14.839625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.840349 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.840353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:14.840473 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:14.841067 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:14.841085 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:14.844228 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":235,"threads_started":1}
W20260928 15:58:14.844627 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:14.836512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:53917 (local address 127.22.125.254:38121)
0928 15:58:14.836740 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:14.836747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:14.836914 (+ 167us) server_negotiation.cc:407] Connection header received
0928 15:58:14.837033 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:14.837036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:14.837097 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:14.837178 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:14.837881 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.838591 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.839731 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:14.840221 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:14.840314 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:14.844410 (+ 4096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:53917: BlockingRecv error: recv got EOF from 127.22.125.194:53917 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 28 15:58:15 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:15 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.195@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:15.294376 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:15.285777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.195:51325)
0928 15:58:15.286431 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:15.286447 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:15.286556 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:15.286910 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:15.286915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:15.287322 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:15.287510 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:15.287519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.288620 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.288628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:15.289545 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:15.289556 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.290274 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.290280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:15.290467 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:15.291137 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:15.291155 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:15.294032 (+ 2877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":364,"threads_started":1}
W20260928 15:58:15.294497 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:15.285839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:51325 (local address 127.22.125.254:38121)
0928 15:58:15.286080 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:15.286104 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:15.286515 (+ 411us) server_negotiation.cc:407] Connection header received
0928 15:58:15.286658 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:15.286663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:15.286744 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:15.286882 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:15.287665 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.288474 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.289691 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.290124 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.290261 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:15.294256 (+ 3995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.195:51325: BlockingRecv error: recv got EOF from 127.22.125.195:51325 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 28 15:58:15 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:15 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.193@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:15.388449 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:15.380413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.193:42135)
0928 15:58:15.380919 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:15.380934 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:15.381042 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:15.381348 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:15.381351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:15.381726 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:15.381888 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:15.381897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.382862 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.382866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:15.383766 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:15.383777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.384484 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.384490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:15.384678 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:15.385320 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:15.385338 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:15.388082 (+ 2744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":259,"threads_started":1}
W20260928 15:58:15.388569 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:15.380449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:42135 (local address 127.22.125.254:38121)
0928 15:58:15.380662 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:15.380672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:15.381016 (+ 344us) server_negotiation.cc:407] Connection header received
0928 15:58:15.381139 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:15.381143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:15.381223 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:15.381303 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:15.382008 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.382742 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.383885 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.384305 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.384453 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:15.388305 (+ 3852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.193:42135: BlockingRecv error: recv got EOF from 127.22.125.193:42135 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 28 15:58:15 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 15:58:15 dist-test-slave-9cqd krb5kdc[31339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.125.194@KRBTEST.COM for oryx/127.22.125.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 15:58:15.858975 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:15.845664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:38121 (local address 127.22.125.194:37389)
0928 15:58:15.846159 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:15.846175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:15.846320 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:15.846653 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:15.846658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:15.847090 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:15.847249 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:15.847258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.848357 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.848367 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:15.849265 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:15.849276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.850033 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.850039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:15.850201 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:15.855774 (+ 5573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:15.855792 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:15.858745 (+ 2953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.125.254:38121: Server oryx/127.22.125.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":266,"threads_started":1}
W20260928 15:58:15.859175 32077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:15.845743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:37389 (local address 127.22.125.254:38121)
0928 15:58:15.845972 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:15.845980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:15.846255 (+ 275us) server_negotiation.cc:407] Connection header received
0928 15:58:15.846448 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:15.846453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:15.846525 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:15.846624 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:15.847392 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.848207 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.849412 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:15.849878 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:15.850013 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:15.858958 (+ 8945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.125.194:37389: BlockingRecv error: recv got EOF from 127.22.125.194:37389 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 15:58:16.143113 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31928
I20260928 15:58:16.154971 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:16.144157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.22.125.193:46203)
0928 15:58:16.144763 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:16.144769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:16.144797 (+ 28us) server_negotiation.cc:407] Connection header received
0928 15:58:16.144853 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:16.144862 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:16.145081 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:16.145229 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:16.146596 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.147611 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.148779 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.149317 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.149425 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:16.151731 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:16.151739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:16.151753 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:16.151783 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:16.154302 (+ 2519us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.125.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 15:58:16.154656 (+ 354us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54236: Request ticket server kudu/127.22.125.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":328,"threads_started":1}
W20260928 15:58:16.155050 32388 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54236: Request ticket server kudu/127.22.125.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 15:58:16.155032 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31928: Not authorized: Client connection negotiation failed: client connection to 127.22.125.193:46203: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.125.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 15:58:16.155381 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 31928
I20260928 15:58:16.172271 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32079
I20260928 15:58:16.183419 32389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:16.173073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.22.125.194:42743)
0928 15:58:16.173661 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:16.173668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:16.173697 (+ 29us) server_negotiation.cc:407] Connection header received
0928 15:58:16.173755 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:16.173761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:16.173981 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:16.174144 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:16.175301 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.176439 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.177686 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.178214 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.178317 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:16.180673 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:16.180681 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:16.180694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:16.180723 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:16.182939 (+ 2216us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.125.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 15:58:16.183188 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37666: Request ticket server kudu/127.22.125.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":265,"threads_started":1}
W20260928 15:58:16.183552 32389 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37666: Request ticket server kudu/127.22.125.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 15:58:16.183548 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32079: Not authorized: Client connection negotiation failed: client connection to 127.22.125.194:42743: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.125.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 15:58:16.183827 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 32079
I20260928 15:58:16.200809 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32210
I20260928 15:58:16.213495 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:16.201727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.22.125.195:42629)
0928 15:58:16.202341 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:16.202348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:16.202373 (+ 25us) server_negotiation.cc:407] Connection header received
0928 15:58:16.202431 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:16.202437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:16.202668 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:16.202816 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:16.204275 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.205343 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.207090 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.207629 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.207745 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:16.210159 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:16.210167 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:16.210180 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:16.210212 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:16.212832 (+ 2620us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.125.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 15:58:16.213121 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48512: Request ticket server kudu/127.22.125.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":334,"threads_started":1}
W20260928 15:58:16.213495 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32210: Not authorized: Client connection negotiation failed: client connection to 127.22.125.195:42629: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.125.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 15:58:16.213583 32390 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48512: Request ticket server kudu/127.22.125.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 15:58:16.213647 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 32210
I20260928 15:58:16.229874 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 31859
I20260928 15:58:16.231069 31921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:16.347431 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 31859
2026-09-28T15:58:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20928 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T15:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:58:16Z Disabled control of system clock
I20260928 15:58:16.421216 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35569
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:35569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 15:58:16.744298 32395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:16.744596 32395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:16.744675 32395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:16.751888 32395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:58:16.751999 32395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:16.752053 32395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:16.752108 32395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:58:16.752146 32395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:58:16.761214 32395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:35569
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:16.762434 32395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:16.764184 32395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:16.775743 32401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:16.775909 32400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:16.776358 32403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:16.776638 32395 server_base.cc:1034] running on GCE node
I20260928 15:58:16.777566 32395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:16.779208 32395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:16.780408 32395 hybrid_clock.cc:648] HybridClock initialized: now 1790611096780364 us; error 44 us; skew 500 ppm
I20260928 15:58:16.780771 32395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:16.782943 32395 webserver.cc:460] Webserver started at http://127.22.125.254:34825/ using document root <none> and password file <none>
I20260928 15:58:16.783633 32395 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:16.783749 32395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:16.784140 32395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:16.787258 32395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "b8da744d5a1748a285a8ce639e420825"
format_stamp: "Formatted at 2026-09-28 15:58:16 on dist-test-slave-9cqd"
server_key: "49b3bb122522dd69d57fa3462288ae95"
server_key_iv: "850247c179bb933b46866f18a0176f20"
server_key_version: "encryptionkey@0"
I20260928 15:58:16.788219 32395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "b8da744d5a1748a285a8ce639e420825"
format_stamp: "Formatted at 2026-09-28 15:58:16 on dist-test-slave-9cqd"
server_key: "49b3bb122522dd69d57fa3462288ae95"
server_key_iv: "850247c179bb933b46866f18a0176f20"
server_key_version: "encryptionkey@0"
I20260928 15:58:16.793844 32395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260928 15:58:16.797379 32409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:16.798811 32395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:58:16.799006 32395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "b8da744d5a1748a285a8ce639e420825"
format_stamp: "Formatted at 2026-09-28 15:58:16 on dist-test-slave-9cqd"
server_key: "49b3bb122522dd69d57fa3462288ae95"
server_key_iv: "850247c179bb933b46866f18a0176f20"
server_key_version: "encryptionkey@0"
I20260928 15:58:16.799232 32395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:16.821262 32395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:16.822139 32395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:16.822436 32395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:16.840510 32395 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:35569
I20260928 15:58:16.840534 32460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:35569 every 8 connection(s)
I20260928 15:58:16.841828 32395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:58:16.845552 32461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:58:16.849432 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 32395
I20260928 15:58:16.849651 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:58:16.850299 23031 external_mini_cluster.cc:1428] Setting key 639991380f08f743ff55896c08a284bf
I20260928 15:58:16.856556 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825: Bootstrap starting.
I20260928 15:58:16.859474 32461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825: Neither blocks nor log segments found. Creating new log.
I20260928 15:58:16.860563 32461 log.cc:824] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825: Log is configured to *not* fsync() on all Append() calls
I20260928 15:58:16.863765 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825: No bootstrap required, opened a new log
I20260928 15:58:16.864059 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:16.852482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.22.125.254:35569)
0928 15:58:16.853445 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:16.853462 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:16.853504 (+ 42us) server_negotiation.cc:407] Connection header received
0928 15:58:16.855206 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:16.855226 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:16.855648 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:16.856015 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:16.856641 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.857934 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.859221 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:16.859993 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:16.860228 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:16.861413 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:16.861421 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:16.861434 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:16.862030 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:16.862143 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:16.862169 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:16.862286 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:16.862491 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 15:58:16.862816 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":339,"threads_started":1}
I20260928 15:58:16.871063 32461 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da744d5a1748a285a8ce639e420825" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35569 } }
I20260928 15:58:16.871438 32461 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:58:16.871572 32461 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8da744d5a1748a285a8ce639e420825, State: Initialized, Role: FOLLOWER
I20260928 15:58:16.872201 32461 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da744d5a1748a285a8ce639e420825" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35569 } }
I20260928 15:58:16.872466 32461 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:58:16.872607 32461 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:58:16.872793 32461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:58:16.874840 32461 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da744d5a1748a285a8ce639e420825" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35569 } }
I20260928 15:58:16.875351 32461 leader_election.cc:302] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8da744d5a1748a285a8ce639e420825; no voters:
I20260928 15:58:16.875911 32461 leader_election.cc:288] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:58:16.876078 32466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:58:16.876758 32466 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [term 1 LEADER]: Becoming Leader. State: Replica: b8da744d5a1748a285a8ce639e420825, State: Running, Role: LEADER
I20260928 15:58:16.877255 32466 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da744d5a1748a285a8ce639e420825" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35569 } }
I20260928 15:58:16.877907 32461 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:58:16.879820 32468 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8da744d5a1748a285a8ce639e420825. Latest consensus state: current_term: 1 leader_uuid: "b8da744d5a1748a285a8ce639e420825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da744d5a1748a285a8ce639e420825" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35569 } } }
I20260928 15:58:16.880244 32468 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [sys.catalog]: This master's current role is: LEADER
I20260928 15:58:16.880431 32467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8da744d5a1748a285a8ce639e420825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da744d5a1748a285a8ce639e420825" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35569 } } }
I20260928 15:58:16.880811 32467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825 [sys.catalog]: This master's current role is: LEADER
I20260928 15:58:16.882664 32471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:58:16.888104 32471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:58:16.896677 32471 catalog_manager.cc:1347] Generated new cluster ID: c9c5d99dbd7a43ceb97693e1fc50b8f3
I20260928 15:58:16.896795 32471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:58:16.906308 32471 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:58:16.907975 32471 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:58:16.920183 32471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8da744d5a1748a285a8ce639e420825: Generated new TSK 0
I20260928 15:58:16.921154 32471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 15:58:16.932762 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35569
--builtin_ntp_servers=127.22.125.212:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 15:58:17.257905 32485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:17.258160 32485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:17.258320 32485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:17.265384 32485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:17.265491 32485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:17.265666 32485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:58:17.274232 32485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:17.275473 32485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:17.277231 32485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:17.290623 32490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:17.290580 32491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:17.290546 32493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:17.291363 32485 server_base.cc:1034] running on GCE node
I20260928 15:58:17.292016 32485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:17.293037 32485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:17.294282 32485 hybrid_clock.cc:648] HybridClock initialized: now 1790611097294225 us; error 46 us; skew 500 ppm
I20260928 15:58:17.294642 32485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:17.297070 32485 webserver.cc:460] Webserver started at http://127.22.125.193:43215/ using document root <none> and password file <none>
I20260928 15:58:17.297756 32485 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:17.297883 32485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:17.298213 32485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:17.301045 32485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "cdd35b2621014f708e15b8c18f3ba921"
format_stamp: "Formatted at 2026-09-28 15:58:17 on dist-test-slave-9cqd"
server_key: "17852b66d733a0f72e31bde8d551fa9d"
server_key_iv: "fdc0651873d327c9dcf888bb709653ca"
server_key_version: "encryptionkey@0"
I20260928 15:58:17.301810 32485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "cdd35b2621014f708e15b8c18f3ba921"
format_stamp: "Formatted at 2026-09-28 15:58:17 on dist-test-slave-9cqd"
server_key: "17852b66d733a0f72e31bde8d551fa9d"
server_key_iv: "fdc0651873d327c9dcf888bb709653ca"
server_key_version: "encryptionkey@0"
I20260928 15:58:17.307181 32485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 15:58:17.310441 32499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:17.311657 32485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260928 15:58:17.311834 32485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "cdd35b2621014f708e15b8c18f3ba921"
format_stamp: "Formatted at 2026-09-28 15:58:17 on dist-test-slave-9cqd"
server_key: "17852b66d733a0f72e31bde8d551fa9d"
server_key_iv: "fdc0651873d327c9dcf888bb709653ca"
server_key_version: "encryptionkey@0"
I20260928 15:58:17.312049 32485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:17.329372 32485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:17.330180 32485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:17.330497 32485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:17.331413 32485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:17.332815 32485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:17.332942 32485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:17.333071 32485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:17.333163 32485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:17.365792 32485 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:36397
I20260928 15:58:17.365851 32611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:36397 every 8 connection(s)
I20260928 15:58:17.367177 32485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:58:17.376768 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 32485
I20260928 15:58:17.376996 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:58:17.377673 23031 external_mini_cluster.cc:1428] Setting key 3daf014cfd198add041b97c2ff7bd0b7
I20260928 15:58:17.379959 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:17.369701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:50811 (local address 127.22.125.254:35569)
0928 15:58:17.370159 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:17.370163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:17.371367 (+ 1204us) server_negotiation.cc:407] Connection header received
0928 15:58:17.373275 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:17.373280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:17.373379 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:17.373499 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:17.374548 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.375261 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.376899 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.377342 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.377481 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:17.379019 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:17.379026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:17.379037 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:17.379055 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:17.379150 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:17.379166 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:17.379287 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:17.379537 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 15:58:17.379611 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":249,"threads_started":1}
I20260928 15:58:17.380661 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35569
--builtin_ntp_servers=127.22.125.212:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 15:58:17.381094 32615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:17.370157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35569 (local address 127.22.125.193:50811)
0928 15:58:17.371195 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:17.371245 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:17.373002 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:17.373733 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:17.373741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:17.373825 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:17.374378 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:17.374394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.375428 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.375433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:17.376745 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:17.376757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.377601 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.377608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:17.377923 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:17.378684 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:58:17.378687 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:17.378744 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:58:17.378825 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:17.379284 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:17.379297 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:17.379307 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 15:58:17.379512 (+ 205us) client_negotiation.cc:240] Negotiation successful
0928 15:58:17.379869 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":271,"threads_started":1}
I20260928 15:58:17.382671 32612 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35569
I20260928 15:58:17.382985 32612 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:17.383818 32612 heartbeater.cc:511] Master 127.22.125.254:35569 requested a full tablet report, sending...
I20260928 15:58:17.385794 32426 ts_manager.cc:194] Registered new tserver with Master: cdd35b2621014f708e15b8c18f3ba921 (127.22.125.193:36397)
I20260928 15:58:17.387255 32426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.193:50811
W20260928 15:58:17.695406 32616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:17.695732 32616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:17.695896 32616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:17.703182 32616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:17.703305 32616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:17.703485 32616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:58:17.712356 32616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:17.713583 32616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:17.715406 32616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:17.728425 32621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:17.728329 32622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:17.728370 32624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:17.729298 32616 server_base.cc:1034] running on GCE node
I20260928 15:58:17.729892 32616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:17.730808 32616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:17.732026 32616 hybrid_clock.cc:648] HybridClock initialized: now 1790611097731968 us; error 56 us; skew 500 ppm
I20260928 15:58:17.732388 32616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:17.734452 32616 webserver.cc:460] Webserver started at http://127.22.125.194:35615/ using document root <none> and password file <none>
I20260928 15:58:17.735136 32616 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:17.735247 32616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:17.735567 32616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:17.738289 32616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "fc4757cf84124dfdbf4a24453cab341f"
format_stamp: "Formatted at 2026-09-28 15:58:17 on dist-test-slave-9cqd"
server_key: "6c8f6fc9169a2420365c736bffab4c5a"
server_key_iv: "a41986795184d6e60857af13cc593382"
server_key_version: "encryptionkey@0"
I20260928 15:58:17.739054 32616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "fc4757cf84124dfdbf4a24453cab341f"
format_stamp: "Formatted at 2026-09-28 15:58:17 on dist-test-slave-9cqd"
server_key: "6c8f6fc9169a2420365c736bffab4c5a"
server_key_iv: "a41986795184d6e60857af13cc593382"
server_key_version: "encryptionkey@0"
I20260928 15:58:17.743970 32616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260928 15:58:17.747283 32630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:17.748658 32616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 15:58:17.748836 32616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "fc4757cf84124dfdbf4a24453cab341f"
format_stamp: "Formatted at 2026-09-28 15:58:17 on dist-test-slave-9cqd"
server_key: "6c8f6fc9169a2420365c736bffab4c5a"
server_key_iv: "a41986795184d6e60857af13cc593382"
server_key_version: "encryptionkey@0"
I20260928 15:58:17.749053 32616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:17.768513 32616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:17.769297 32616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:17.769605 32616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:17.770531 32616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:17.771952 32616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:17.772109 32616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:17.772238 32616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:17.772328 32616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:17.807731 32616 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:34861
I20260928 15:58:17.807761 32742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:34861 every 8 connection(s)
I20260928 15:58:17.809098 32616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:58:17.812418 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 32616
I20260928 15:58:17.812614 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:58:17.813051 23031 external_mini_cluster.cc:1428] Setting key 46a545e33cb00e0a1c765941d5816670
I20260928 15:58:17.815883 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35569
--builtin_ntp_servers=127.22.125.212:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 15:58:17.822062 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:17.811661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:35739 (local address 127.22.125.254:35569)
0928 15:58:17.811841 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:17.811845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:17.813448 (+ 1603us) server_negotiation.cc:407] Connection header received
0928 15:58:17.815535 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:17.815540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:17.815631 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:17.815742 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:17.817039 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.817744 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.819295 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.819691 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.819849 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:17.821253 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:17.821257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:17.821260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:17.821277 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:17.821311 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:17.821322 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:17.821427 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:17.821651 (+ 224us) server_negotiation.cc:299] Negotiation successful
0928 15:58:17.821750 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 15:58:17.823050 32745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:17.812206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35569 (local address 127.22.125.194:35739)
0928 15:58:17.813262 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:17.813314 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:17.815184 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:17.815986 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:17.816017 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:17.816158 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:17.816875 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:17.816894 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.817909 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.817914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:17.819141 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:17.819151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:17.819930 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:17.819937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:17.820235 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:17.820932 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:58:17.820935 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:17.820972 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:58:17.821049 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:17.821426 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:17.821438 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:17.821448 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 15:58:17.821623 (+ 175us) client_negotiation.cc:240] Negotiation successful
0928 15:58:17.822026 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":334,"threads_started":1}
I20260928 15:58:17.824501 32743 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35569
I20260928 15:58:17.824796 32743 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:17.825516 32743 heartbeater.cc:511] Master 127.22.125.254:35569 requested a full tablet report, sending...
I20260928 15:58:17.826969 32426 ts_manager.cc:194] Registered new tserver with Master: fc4757cf84124dfdbf4a24453cab341f (127.22.125.194:34861)
I20260928 15:58:17.828222 32426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.194:35739
W20260928 15:58:18.140985 32746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:18.141270 32746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:18.141361 32746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:18.148456 32746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:18.148550 32746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:18.148651 32746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:58:18.157684 32746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:18.158871 32746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:18.160604 32746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:18.173569 32751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:18.173527 32754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:18.173619 32752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:18.174693 32746 server_base.cc:1034] running on GCE node
I20260928 15:58:18.175304 32746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:18.176283 32746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:18.177508 32746 hybrid_clock.cc:648] HybridClock initialized: now 1790611098177471 us; error 48 us; skew 500 ppm
I20260928 15:58:18.177817 32746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:18.179975 32746 webserver.cc:460] Webserver started at http://127.22.125.195:45195/ using document root <none> and password file <none>
I20260928 15:58:18.180703 32746 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:18.180843 32746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:18.181164 32746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:18.184114 32746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "04427bd0e8c14283baf6e9bc98328be6"
format_stamp: "Formatted at 2026-09-28 15:58:18 on dist-test-slave-9cqd"
server_key: "118686ad9b31eeb1800376d33ad1605f"
server_key_iv: "d4bdfa41da830f88bb8edd1e08c0ad43"
server_key_version: "encryptionkey@0"
I20260928 15:58:18.184935 32746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "04427bd0e8c14283baf6e9bc98328be6"
format_stamp: "Formatted at 2026-09-28 15:58:18 on dist-test-slave-9cqd"
server_key: "118686ad9b31eeb1800376d33ad1605f"
server_key_iv: "d4bdfa41da830f88bb8edd1e08c0ad43"
server_key_version: "encryptionkey@0"
I20260928 15:58:18.190019 32746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:58:18.193571 32760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:18.194980 32746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:58:18.195182 32746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "04427bd0e8c14283baf6e9bc98328be6"
format_stamp: "Formatted at 2026-09-28 15:58:18 on dist-test-slave-9cqd"
server_key: "118686ad9b31eeb1800376d33ad1605f"
server_key_iv: "d4bdfa41da830f88bb8edd1e08c0ad43"
server_key_version: "encryptionkey@0"
I20260928 15:58:18.195420 32746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:18.214480 32746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:18.215291 32746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:18.215598 32746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:18.216691 32746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:18.218122 32746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:18.218242 32746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:18.218379 32746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:18.218483 32746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:18.251073 32746 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:45913
I20260928 15:58:18.251125 405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:45913 every 8 connection(s)
I20260928 15:58:18.252506 32746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:58:18.258669 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 32746
I20260928 15:58:18.258872 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:58:18.259294 23031 external_mini_cluster.cc:1428] Setting key 3bacac87b11bc49baa295cf910fb4a75
I20260928 15:58:18.265450 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:18.255017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:57975 (local address 127.22.125.254:35569)
0928 15:58:18.255193 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:18.255198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:18.256646 (+ 1448us) server_negotiation.cc:407] Connection header received
0928 15:58:18.258396 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:18.258401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:18.258496 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:18.258611 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:18.259847 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.260850 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.262724 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.263133 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.263271 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:18.264614 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:18.264619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:18.264624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:18.264650 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:18.264682 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:18.264693 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:18.264793 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:18.265019 (+ 226us) server_negotiation.cc:299] Negotiation successful
0928 15:58:18.265117 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 15:58:18.266440 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:18.255549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35569 (local address 127.22.125.195:57975)
0928 15:58:18.256486 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:18.256530 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:18.258119 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:18.258886 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:18.258895 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:18.259017 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:18.259626 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:18.259644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.260997 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.261003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:18.262570 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:18.262582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.263365 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.263373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:18.263599 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:18.264311 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:58:18.264314 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:18.264352 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:58:18.264432 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:18.264790 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:18.264802 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:18.264811 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 15:58:18.265008 (+ 197us) client_negotiation.cc:240] Negotiation successful
0928 15:58:18.265397 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":271,"threads_started":1}
I20260928 15:58:18.267693 406 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35569
I20260928 15:58:18.267990 406 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:18.268748 406 heartbeater.cc:511] Master 127.22.125.254:35569 requested a full tablet report, sending...
I20260928 15:58:18.270015 32426 ts_manager.cc:194] Registered new tserver with Master: 04427bd0e8c14283baf6e9bc98328be6 (127.22.125.195:45913)
I20260928 15:58:18.270787 32426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.195:57975
I20260928 15:58:18.274329 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 15:58:18.280606 32614 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:18.278901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.22.125.254:35569)
0928 15:58:18.279083 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:18.279087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:18.279559 (+ 472us) server_negotiation.cc:407] Connection header received
0928 15:58:18.279763 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:18.279767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:18.279835 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:18.279942 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:18.280434 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46998: BlockingRecv error: recv got EOF from 127.0.0.1:46998 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 15:58:18.281551 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32485
I20260928 15:58:18.289242 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:18.282424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.22.125.193:36397)
0928 15:58:18.282970 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:18.282976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:18.283000 (+ 24us) server_negotiation.cc:407] Connection header received
0928 15:58:18.283053 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:18.283059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:18.283286 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:18.283465 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:18.283794 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.284876 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.286161 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.286853 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.286960 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:18.287874 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:18.287882 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:18.287894 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:18.288554 (+ 660us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:18.288601 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:18.288621 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:18.288711 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:18.288837 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 15:58:18.288963 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":269,"threads_started":1}
I20260928 15:58:18.290668 32607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:18.373951 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 32485
I20260928 15:58:18.388263 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32616
I20260928 15:58:18.396377 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:18.389168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54796 (local address 127.22.125.194:34861)
0928 15:58:18.389676 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:18.389684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:18.389710 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:58:18.389769 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:18.389776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:18.389995 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:18.390169 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:18.390605 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.391715 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.393141 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.393704 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.393844 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:18.394876 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:18.394885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:18.394898 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:18.395488 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:18.395534 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:18.395552 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:18.395680 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:18.395855 (+ 175us) server_negotiation.cc:299] Negotiation successful
0928 15:58:18.395984 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":251,"threads_started":1}
I20260928 15:58:18.397591 32738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:18.478406 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 32616
I20260928 15:58:18.494645 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32746
I20260928 15:58:18.503324 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:18.495606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.22.125.195:45913)
0928 15:58:18.496349 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:18.496379 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:18.496425 (+ 46us) server_negotiation.cc:407] Connection header received
0928 15:58:18.496508 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:18.496521 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:18.496864 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:18.497058 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:18.497587 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.498783 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.500011 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:18.500721 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:18.500820 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:18.501836 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:18.501843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:18.501858 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:18.502491 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:18.502537 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:18.502558 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:18.502645 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:18.502787 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 15:58:18.502944 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":320,"threads_started":1}
I20260928 15:58:18.504711 400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:18.592398 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 32746
I20260928 15:58:18.609643 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 32395
I20260928 15:58:18.610852 32456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:18.714243 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 32395
2026-09-28T15:58:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2361 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T15:58:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:58:18Z Disabled control of system clock
I20260928 15:58:18.786415 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39621
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:39621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 15:58:19.107543 425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:19.107887 425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:19.108084 425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:19.115396 425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:58:19.115548 425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:19.115640 425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:19.115732 425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:58:19.115799 425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:58:19.125101 425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:39621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:19.126421 425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:19.128310 425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:19.140210 430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:19.140167 431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:19.140249 433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:19.140931 425 server_base.cc:1034] running on GCE node
I20260928 15:58:19.141757 425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:19.143134 425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:19.144371 425 hybrid_clock.cc:648] HybridClock initialized: now 1790611099144333 us; error 44 us; skew 500 ppm
I20260928 15:58:19.144748 425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:19.146930 425 webserver.cc:460] Webserver started at http://127.22.125.254:39499/ using document root <none> and password file <none>
I20260928 15:58:19.147629 425 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:19.147759 425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:19.148150 425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:19.150961 425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "18dc4ee74647476e8995f5bb14ca0ddf"
format_stamp: "Formatted at 2026-09-28 15:58:19 on dist-test-slave-9cqd"
server_key: "9f818556461fa95168dfcaca7d9b02a5"
server_key_iv: "10b2a69e10402fbced661c85700d25db"
server_key_version: "encryptionkey@0"
I20260928 15:58:19.151765 425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "18dc4ee74647476e8995f5bb14ca0ddf"
format_stamp: "Formatted at 2026-09-28 15:58:19 on dist-test-slave-9cqd"
server_key: "9f818556461fa95168dfcaca7d9b02a5"
server_key_iv: "10b2a69e10402fbced661c85700d25db"
server_key_version: "encryptionkey@0"
I20260928 15:58:19.157089 425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 15:58:19.160477 439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:19.161783 425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260928 15:58:19.161979 425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "18dc4ee74647476e8995f5bb14ca0ddf"
format_stamp: "Formatted at 2026-09-28 15:58:19 on dist-test-slave-9cqd"
server_key: "9f818556461fa95168dfcaca7d9b02a5"
server_key_iv: "10b2a69e10402fbced661c85700d25db"
server_key_version: "encryptionkey@0"
I20260928 15:58:19.162205 425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:19.167938 425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:19.168746 425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:19.169047 425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:19.186906 425 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:39621
I20260928 15:58:19.186941 490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:39621 every 8 connection(s)
I20260928 15:58:19.188416 425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:58:19.192032 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:58:19.194815 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 425
I20260928 15:58:19.194999 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:58:19.195784 23031 external_mini_cluster.cc:1428] Setting key b5abaf7c6c35837b42f5e0e057b1288f
I20260928 15:58:19.204380 491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf: Bootstrap starting.
I20260928 15:58:19.204399 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:19.198061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.22.125.254:39621)
0928 15:58:19.199017 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:19.199028 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:19.199070 (+ 42us) server_negotiation.cc:407] Connection header received
0928 15:58:19.200802 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:19.200825 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:19.201250 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:19.201655 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:19.202760 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:19.202766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:19.202778 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:19.202791 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:19.202842 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:19.202866 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:19.203089 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:19.203218 (+ 129us) server_negotiation.cc:299] Negotiation successful
0928 15:58:19.203472 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":358,"threads_started":1}
I20260928 15:58:19.207481 491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf: Neither blocks nor log segments found. Creating new log.
I20260928 15:58:19.208524 491 log.cc:824] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf: Log is configured to *not* fsync() on all Append() calls
I20260928 15:58:19.210978 491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf: No bootstrap required, opened a new log
I20260928 15:58:19.218472 491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39621 } }
I20260928 15:58:19.218861 491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:58:19.218979 491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18dc4ee74647476e8995f5bb14ca0ddf, State: Initialized, Role: FOLLOWER
I20260928 15:58:19.219520 491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39621 } }
I20260928 15:58:19.219750 491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:58:19.219919 491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:58:19.220093 491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:58:19.222133 491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39621 } }
I20260928 15:58:19.222612 491 leader_election.cc:302] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18dc4ee74647476e8995f5bb14ca0ddf; no voters:
I20260928 15:58:19.223265 491 leader_election.cc:288] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:58:19.223418 497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:58:19.225303 491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:58:19.227823 497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [term 1 LEADER]: Becoming Leader. State: Replica: 18dc4ee74647476e8995f5bb14ca0ddf, State: Running, Role: LEADER
I20260928 15:58:19.228440 497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39621 } }
I20260928 15:58:19.231526 498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39621 } } }
I20260928 15:58:19.231846 498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [sys.catalog]: This master's current role is: LEADER
I20260928 15:58:19.232275 499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18dc4ee74647476e8995f5bb14ca0ddf. Latest consensus state: current_term: 1 leader_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dc4ee74647476e8995f5bb14ca0ddf" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39621 } } }
I20260928 15:58:19.233187 499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf [sys.catalog]: This master's current role is: LEADER
W20260928 15:58:19.236150 511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 15:58:19.236272 511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 15:58:19.236413 512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:58:19.237964 512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:58:19.246523 512 catalog_manager.cc:1347] Generated new cluster ID: 5daabee76a8e4fcb8bd44292ac46b08f
I20260928 15:58:19.246631 512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:58:19.259099 512 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:58:19.259598 512 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:58:19.267819 512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18dc4ee74647476e8995f5bb14ca0ddf: Generated new TSK 0
I20260928 15:58:19.268782 512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 15:58:19.273998 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39621
--builtin_ntp_servers=127.22.125.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 15:58:19.602177 516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:19.602450 516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:19.602540 516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:19.609587 516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:19.609684 516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:19.609790 516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:58:19.618537 516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:19.619747 516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:19.621479 516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:19.634512 521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:19.634480 522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:19.634431 524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:19.635268 516 server_base.cc:1034] running on GCE node
I20260928 15:58:19.635789 516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:19.636696 516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:19.637925 516 hybrid_clock.cc:648] HybridClock initialized: now 1790611099637876 us; error 60 us; skew 500 ppm
I20260928 15:58:19.638254 516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:19.640558 516 webserver.cc:460] Webserver started at http://127.22.125.193:38165/ using document root <none> and password file <none>
I20260928 15:58:19.641237 516 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:19.641366 516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:19.641808 516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:19.644963 516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "b93ec56cef2442d0b70175dec7727684"
format_stamp: "Formatted at 2026-09-28 15:58:19 on dist-test-slave-9cqd"
server_key: "7e87b8d367338e41a06ad064b699ba19"
server_key_iv: "f6ceee2d86ffc1f20f652efbaf8aaad0"
server_key_version: "encryptionkey@0"
I20260928 15:58:19.645742 516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "b93ec56cef2442d0b70175dec7727684"
format_stamp: "Formatted at 2026-09-28 15:58:19 on dist-test-slave-9cqd"
server_key: "7e87b8d367338e41a06ad064b699ba19"
server_key_iv: "f6ceee2d86ffc1f20f652efbaf8aaad0"
server_key_version: "encryptionkey@0"
I20260928 15:58:19.650682 516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:58:19.653970 530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:19.655339 516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 15:58:19.655507 516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "b93ec56cef2442d0b70175dec7727684"
format_stamp: "Formatted at 2026-09-28 15:58:19 on dist-test-slave-9cqd"
server_key: "7e87b8d367338e41a06ad064b699ba19"
server_key_iv: "f6ceee2d86ffc1f20f652efbaf8aaad0"
server_key_version: "encryptionkey@0"
I20260928 15:58:19.655719 516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:19.661230 516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:19.661993 516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:19.662268 516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:19.663127 516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:19.664527 516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:19.664649 516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:19.664772 516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:19.664870 516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:19.701457 516 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:38315
I20260928 15:58:19.701548 643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:38315 every 8 connection(s)
I20260928 15:58:19.703023 516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:58:19.705896 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 516
I20260928 15:58:19.706111 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:58:19.706933 23031 external_mini_cluster.cc:1428] Setting key 54ad92f94d19a46b8a40fa4e9cb39033
I20260928 15:58:19.710045 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39621
--builtin_ntp_servers=127.22.125.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 15:58:19.712163 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:19.705811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:38841 (local address 127.22.125.254:39621)
0928 15:58:19.706400 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:19.706407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:19.707729 (+ 1322us) server_negotiation.cc:407] Connection header received
0928 15:58:19.709754 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:19.709760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:19.709824 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:19.709957 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:19.711440 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:19.711444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:19.711447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:19.711462 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:19.711504 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:19.711516 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:19.711609 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:19.711836 (+ 227us) server_negotiation.cc:299] Negotiation successful
0928 15:58:19.711886 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":282,"threads_started":1}
I20260928 15:58:19.713400 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:19.706445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39621 (local address 127.22.125.193:38841)
0928 15:58:19.707557 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:19.707614 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:19.709425 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:19.710236 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:19.710252 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:19.710387 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:19.711085 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:58:19.711091 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:19.711144 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:58:19.711264 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:19.711619 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:19.711632 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:19.711645 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 15:58:19.711912 (+ 267us) client_negotiation.cc:240] Negotiation successful
0928 15:58:19.712197 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":298,"threads_started":1}
I20260928 15:58:19.714851 644 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39621
I20260928 15:58:19.715142 644 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:19.715704 644 heartbeater.cc:511] Master 127.22.125.254:39621 requested a full tablet report, sending...
I20260928 15:58:19.717859 456 ts_manager.cc:194] Registered new tserver with Master: b93ec56cef2442d0b70175dec7727684 (127.22.125.193:38315)
W20260928 15:58:20.031536 648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:20.031798 648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:20.031884 648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:20.038983 648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:20.039080 648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:20.039196 648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:58:20.048208 648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:20.049405 648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:20.051141 648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:20.063923 653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:20.064000 654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:20.064020 656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:20.065187 648 server_base.cc:1034] running on GCE node
I20260928 15:58:20.065814 648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:20.066831 648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:20.068024 648 hybrid_clock.cc:648] HybridClock initialized: now 1790611100067988 us; error 32 us; skew 500 ppm
I20260928 15:58:20.068411 648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:20.070617 648 webserver.cc:460] Webserver started at http://127.22.125.194:41487/ using document root <none> and password file <none>
I20260928 15:58:20.071286 648 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:20.071400 648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:20.071729 648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:20.074517 648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "b0d0cb54ebea4861938b0e02cf96c249"
format_stamp: "Formatted at 2026-09-28 15:58:20 on dist-test-slave-9cqd"
server_key: "2733aa17cd0e82a039b62fada60798c7"
server_key_iv: "eb56f80a045c25504c89228c5d4d1138"
server_key_version: "encryptionkey@0"
I20260928 15:58:20.075287 648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "b0d0cb54ebea4861938b0e02cf96c249"
format_stamp: "Formatted at 2026-09-28 15:58:20 on dist-test-slave-9cqd"
server_key: "2733aa17cd0e82a039b62fada60798c7"
server_key_iv: "eb56f80a045c25504c89228c5d4d1138"
server_key_version: "encryptionkey@0"
I20260928 15:58:20.080376 648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260928 15:58:20.083650 662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:20.084949 648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:58:20.085115 648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "b0d0cb54ebea4861938b0e02cf96c249"
format_stamp: "Formatted at 2026-09-28 15:58:20 on dist-test-slave-9cqd"
server_key: "2733aa17cd0e82a039b62fada60798c7"
server_key_iv: "eb56f80a045c25504c89228c5d4d1138"
server_key_version: "encryptionkey@0"
I20260928 15:58:20.085337 648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:20.090764 648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:20.091516 648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:20.091785 648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:20.092728 648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:20.094118 648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:20.094233 648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:20.094359 648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:20.094452 648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:20.126693 648 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:38113
I20260928 15:58:20.126783 775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:38113 every 8 connection(s)
I20260928 15:58:20.128127 648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:58:20.134542 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 648
I20260928 15:58:20.134742 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:58:20.135370 23031 external_mini_cluster.cc:1428] Setting key 0d19803de724a88a139c05878c2db2ed
I20260928 15:58:20.136852 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:20.130784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:40399 (local address 127.22.125.254:39621)
0928 15:58:20.130966 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:20.130970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:20.132397 (+ 1427us) server_negotiation.cc:407] Connection header received
0928 15:58:20.134055 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:20.134061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:20.134138 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:20.134248 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:20.135711 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:20.135715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:20.135718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:20.135734 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:20.135764 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:20.135775 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:20.135911 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:20.136443 (+ 532us) server_negotiation.cc:299] Negotiation successful
0928 15:58:20.136512 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 15:58:20.138242 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39621
--builtin_ntp_servers=127.22.125.212:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 15:58:20.138609 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:20.131285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39621 (local address 127.22.125.194:40399)
0928 15:58:20.132243 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:20.132279 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:20.133793 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:20.134471 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:20.134480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:20.134587 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:20.135387 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:58:20.135390 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:20.135425 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:58:20.135543 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:20.136211 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:20.136222 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:20.136231 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 15:58:20.136523 (+ 292us) client_negotiation.cc:240] Negotiation successful
0928 15:58:20.136766 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":264,"threads_started":1}
I20260928 15:58:20.139961 776 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39621
I20260928 15:58:20.140290 776 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:20.140853 776 heartbeater.cc:511] Master 127.22.125.254:39621 requested a full tablet report, sending...
I20260928 15:58:20.142155 456 ts_manager.cc:194] Registered new tserver with Master: b0d0cb54ebea4861938b0e02cf96c249 (127.22.125.194:38113)
W20260928 15:58:20.455976 779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:20.456300 779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:20.456485 779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:20.463544 779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:20.463625 779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:20.463809 779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:58:20.472582 779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:20.473767 779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:20.475471 779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:20.492399 785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:20.492398 787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:20.492430 784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:20.493124 779 server_base.cc:1034] running on GCE node
I20260928 15:58:20.493711 779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:20.494625 779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:20.495863 779 hybrid_clock.cc:648] HybridClock initialized: now 1790611100495801 us; error 58 us; skew 500 ppm
I20260928 15:58:20.496238 779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 15:58:20.498409 779 webserver.cc:460] Webserver started at http://127.22.125.195:44865/ using document root <none> and password file <none>
I20260928 15:58:20.499073 779 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:20.499186 779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:20.499514 779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:20.502331 779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "d10a8ff9184d42ac90422595c3faf0c2"
format_stamp: "Formatted at 2026-09-28 15:58:20 on dist-test-slave-9cqd"
server_key: "b9d52e9e77bde86bf24993c6b98289bc"
server_key_iv: "8e1b74f4fc914327f57552fca1e93dc5"
server_key_version: "encryptionkey@0"
I20260928 15:58:20.503089 779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "d10a8ff9184d42ac90422595c3faf0c2"
format_stamp: "Formatted at 2026-09-28 15:58:20 on dist-test-slave-9cqd"
server_key: "b9d52e9e77bde86bf24993c6b98289bc"
server_key_iv: "8e1b74f4fc914327f57552fca1e93dc5"
server_key_version: "encryptionkey@0"
I20260928 15:58:20.508181 779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 15:58:20.511225 793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:20.512436 779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 15:58:20.512590 779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "d10a8ff9184d42ac90422595c3faf0c2"
format_stamp: "Formatted at 2026-09-28 15:58:20 on dist-test-slave-9cqd"
server_key: "b9d52e9e77bde86bf24993c6b98289bc"
server_key_iv: "8e1b74f4fc914327f57552fca1e93dc5"
server_key_version: "encryptionkey@0"
I20260928 15:58:20.512812 779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:20.518173 779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:20.518890 779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:20.519145 779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:20.519977 779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:20.521687 779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:20.521792 779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:20.521898 779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:20.521971 779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:20.555698 779 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:35697
I20260928 15:58:20.555753 905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:35697 every 8 connection(s)
I20260928 15:58:20.557047 779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:58:20.558945 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 779
I20260928 15:58:20.559121 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:58:20.559602 23031 external_mini_cluster.cc:1428] Setting key 93ff04b45d97c241d863b9ec93a8a396
I20260928 15:58:20.565591 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:20.559884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:37763 (local address 127.22.125.254:39621)
0928 15:58:20.560127 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:20.560135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:20.561513 (+ 1378us) server_negotiation.cc:407] Connection header received
0928 15:58:20.563391 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:20.563396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:20.563484 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:20.563598 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:20.564832 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:20.564838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:20.564842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:20.564869 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:20.564908 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:20.564921 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:20.565045 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:20.565235 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 15:58:20.565305 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 15:58:20.566504 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:20.560496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39621 (local address 127.22.125.195:37763)
0928 15:58:20.561367 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:20.561411 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:20.563131 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:20.563820 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:20.563828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:20.563915 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:20.564551 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 15:58:20.564554 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:20.564590 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 15:58:20.564670 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:20.565013 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:20.565025 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:20.565034 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 15:58:20.565312 (+ 278us) client_negotiation.cc:240] Negotiation successful
0928 15:58:20.565588 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":293,"threads_started":1}
I20260928 15:58:20.567620 906 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39621
I20260928 15:58:20.567890 906 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:20.568523 906 heartbeater.cc:511] Master 127.22.125.254:39621 requested a full tablet report, sending...
I20260928 15:58:20.569768 456 ts_manager.cc:194] Registered new tserver with Master: d10a8ff9184d42ac90422595c3faf0c2 (127.22.125.195:35697)
I20260928 15:58:20.574647 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 15:58:20.580328 646 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 15:58:20.578940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.22.125.254:39621)
0928 15:58:20.579078 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:20.579082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:20.579567 (+ 485us) server_negotiation.cc:407] Connection header received
0928 15:58:20.579680 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:20.579684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:20.579729 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:20.579798 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:20.580108 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38466: BlockingRecv error: recv got EOF from 127.0.0.1:38466 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 15:58:20.581115 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 516
I20260928 15:58:20.584290 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:20.581932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.22.125.193:38315)
0928 15:58:20.582423 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:20.582429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:20.582453 (+ 24us) server_negotiation.cc:407] Connection header received
0928 15:58:20.582518 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:20.582523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:20.582687 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:20.582800 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:20.583681 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:20.583688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:20.583697 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:20.583710 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:20.583751 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:20.583768 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:20.583868 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:20.583945 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 15:58:20.584020 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":219,"threads_started":1}
I20260928 15:58:20.585348 639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:20.665809 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 516
I20260928 15:58:20.681856 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 648
I20260928 15:58:20.685259 917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:20.682765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.22.125.194:38113)
0928 15:58:20.683255 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:20.683263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:20.683289 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:58:20.683351 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:20.683358 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:20.683575 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:20.683705 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:20.684587 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:20.684592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:20.684604 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:20.684616 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:20.684656 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:20.684672 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:20.684778 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:20.684850 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 15:58:20.684982 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":234,"threads_started":1}
I20260928 15:58:20.686455 771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:20.767262 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 648
I20260928 15:58:20.782967 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 779
I20260928 15:58:20.786944 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:20.783994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55646 (local address 127.22.125.195:35697)
0928 15:58:20.784700 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:20.784707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:20.784737 (+ 30us) server_negotiation.cc:407] Connection header received
0928 15:58:20.784799 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:20.784805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:20.785034 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:20.785159 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:20.786209 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:20.786216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:20.786227 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 15:58:20.786244 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:20.786282 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 15:58:20.786295 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:20.786424 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:20.786483 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 15:58:20.786594 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":403,"threads_started":1}
I20260928 15:58:20.788249 901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:20.872149 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 779
I20260928 15:58:20.887600 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 425
I20260928 15:58:20.888974 486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:20.990346 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 425
2026-09-28T15:58:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2275 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:58:21 dist-test-slave-9cqd krb5kdc[924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:58:21 dist-test-slave-9cqd krb5kdc[924](info): set up 2 sockets
Sep 28 15:58:21 dist-test-slave-9cqd krb5kdc[924](info): commencing operation
krb5kdc: starting...
W20260928 15:58:22.090801 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:58:22 dist-test-slave-9cqd krb5kdc[924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:58:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:58:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:58:22.303117 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44759
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:44759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:58:22.637418 937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:22.637727 937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:22.637894 937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:22.644829 937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:58:22.644965 937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:22.645063 937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:22.645166 937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:58:22.645253 937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:58:22.654556 937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:44759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:22.655768 937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:22.657060 937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:22.668380 942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:22.668344 943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:22.668547 945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:22.669502 937 server_base.cc:1034] running on GCE node
I20260928 15:58:22.670301 937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:22.671648 937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:22.672884 937 hybrid_clock.cc:648] HybridClock initialized: now 1790611102672839 us; error 43 us; skew 500 ppm
I20260928 15:58:22.673218 937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:58:22 dist-test-slave-9cqd krb5kdc[924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611102, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:58:22.676306 937 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:58:22.677660 937 webserver.cc:460] Webserver started at http://127.22.125.254:42359/ using document root <none> and password file <none>
I20260928 15:58:22.678299 937 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:22.678452 937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:22.678754 937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:22.681794 937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "291059f6fdeb42319a3158498d233118"
format_stamp: "Formatted at 2026-09-28 15:58:22 on dist-test-slave-9cqd"
server_key: "baaf4528dff6de94b92be5b0b577078d"
server_key_iv: "760e5605b60cedf70912fd7434c79042"
server_key_version: "encryptionkey@0"
I20260928 15:58:22.682567 937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "291059f6fdeb42319a3158498d233118"
format_stamp: "Formatted at 2026-09-28 15:58:22 on dist-test-slave-9cqd"
server_key: "baaf4528dff6de94b92be5b0b577078d"
server_key_iv: "760e5605b60cedf70912fd7434c79042"
server_key_version: "encryptionkey@0"
I20260928 15:58:22.687389 937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260928 15:58:22.690778 952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:22.692130 937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:58:22.692301 937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "291059f6fdeb42319a3158498d233118"
format_stamp: "Formatted at 2026-09-28 15:58:22 on dist-test-slave-9cqd"
server_key: "baaf4528dff6de94b92be5b0b577078d"
server_key_iv: "760e5605b60cedf70912fd7434c79042"
server_key_version: "encryptionkey@0"
I20260928 15:58:22.692490 937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:22.722831 937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:22.727263 937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:22.727586 937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:22.746893 937 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:44759
I20260928 15:58:22.746930 1005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:44759 every 8 connection(s)
I20260928 15:58:22.748379 937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:58:22.752244 1006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:58:22.752905 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 937
I20260928 15:58:22.753137 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:58:22.753854 23031 external_mini_cluster.cc:1428] Setting key 90856f02f5dcf4be9301cf9a9f5d2da7
I20260928 15:58:22.766229 1006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118: Bootstrap starting.
Sep 28 15:58:22 dist-test-slave-9cqd krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:58:22.769563 1006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118: Neither blocks nor log segments found. Creating new log.
I20260928 15:58:22.770750 1006 log.cc:824] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118: Log is configured to *not* fsync() on all Append() calls
I20260928 15:58:22.773761 1006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118: No bootstrap required, opened a new log
I20260928 15:58:22.777462 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:22.755984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.22.125.254:44759)
0928 15:58:22.756852 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:22.756869 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:22.756919 (+ 50us) server_negotiation.cc:407] Connection header received
0928 15:58:22.758788 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:22.758816 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:22.759319 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:22.759753 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:22.761199 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:22.763202 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:22.764620 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:22.765624 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:22.765812 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:22.770116 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:22.770124 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:22.770138 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:22.770201 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:22.773377 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:22.774220 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:22.774226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:22.774231 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:22.774400 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:22.774784 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:22.774787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:22.774789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:22.775302 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:22.775553 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:22.775984 (+ 431us) server_negotiation.cc:299] Negotiation successful
0928 15:58:22.776455 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":323,"threads_started":1}
I20260928 15:58:22.781984 1006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291059f6fdeb42319a3158498d233118" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44759 } }
I20260928 15:58:22.782344 1006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:58:22.782456 1006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291059f6fdeb42319a3158498d233118, State: Initialized, Role: FOLLOWER
I20260928 15:58:22.782989 1006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291059f6fdeb42319a3158498d233118" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44759 } }
I20260928 15:58:22.783212 1006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:58:22.783375 1006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:58:22.783537 1006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:58:22.786074 1006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291059f6fdeb42319a3158498d233118" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44759 } }
I20260928 15:58:22.786563 1006 leader_election.cc:302] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 291059f6fdeb42319a3158498d233118; no voters:
I20260928 15:58:22.787137 1006 leader_election.cc:288] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:58:22.787268 1011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:58:22.787971 1011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [term 1 LEADER]: Becoming Leader. State: Replica: 291059f6fdeb42319a3158498d233118, State: Running, Role: LEADER
I20260928 15:58:22.788533 1011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291059f6fdeb42319a3158498d233118" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44759 } }
I20260928 15:58:22.789186 1006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:58:22.791225 1012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "291059f6fdeb42319a3158498d233118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291059f6fdeb42319a3158498d233118" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44759 } } }
I20260928 15:58:22.791553 1012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [sys.catalog]: This master's current role is: LEADER
I20260928 15:58:22.796739 1013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291059f6fdeb42319a3158498d233118. Latest consensus state: current_term: 1 leader_uuid: "291059f6fdeb42319a3158498d233118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291059f6fdeb42319a3158498d233118" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44759 } } }
I20260928 15:58:22.796996 1013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118 [sys.catalog]: This master's current role is: LEADER
I20260928 15:58:22.802274 1017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:58:22.815145 1017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:58:22.824260 1017 catalog_manager.cc:1347] Generated new cluster ID: 684eff056e5045cfbb29adf1b2009eed
I20260928 15:58:22.824393 1017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:58:22.840726 1017 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:58:22.842537 1017 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:58:22.853796 1017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118: Generated new TSK 0
I20260928 15:58:22.854820 1017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:58:22.906836 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44759
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:58:23.227363 1032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:23.227615 1032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:23.227706 1032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:23.234628 1032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:23.234727 1032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:23.234834 1032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:58:23.243844 1032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:23.245021 1032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:23.246347 1032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:23.258983 1037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:23.259006 1038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:23.258946 1040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:23.260097 1032 server_base.cc:1034] running on GCE node
I20260928 15:58:23.260625 1032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:23.261435 1032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:23.262634 1032 hybrid_clock.cc:648] HybridClock initialized: now 1790611103262591 us; error 40 us; skew 500 ppm
I20260928 15:58:23.262943 1032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:58:23 dist-test-slave-9cqd krb5kdc[924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611103, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:58:23.266037 1032 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:58:23.267457 1032 webserver.cc:460] Webserver started at http://127.22.125.193:46145/ using document root <none> and password file <none>
I20260928 15:58:23.268172 1032 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:23.268317 1032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:23.268623 1032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:23.271378 1032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99"
format_stamp: "Formatted at 2026-09-28 15:58:23 on dist-test-slave-9cqd"
server_key: "7af37047a9e25cc76b5ecd67df9f7bb1"
server_key_iv: "833e9a8551698f74d53a054c01cadcf4"
server_key_version: "encryptionkey@0"
I20260928 15:58:23.272199 1032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99"
format_stamp: "Formatted at 2026-09-28 15:58:23 on dist-test-slave-9cqd"
server_key: "7af37047a9e25cc76b5ecd67df9f7bb1"
server_key_iv: "833e9a8551698f74d53a054c01cadcf4"
server_key_version: "encryptionkey@0"
I20260928 15:58:23.277377 1032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260928 15:58:23.280956 1047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:23.282220 1032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 15:58:23.282402 1032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99"
format_stamp: "Formatted at 2026-09-28 15:58:23 on dist-test-slave-9cqd"
server_key: "7af37047a9e25cc76b5ecd67df9f7bb1"
server_key_iv: "833e9a8551698f74d53a054c01cadcf4"
server_key_version: "encryptionkey@0"
I20260928 15:58:23.282585 1032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:23.302265 1032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:23.306576 1032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:23.306875 1032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:23.307819 1032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:23.309202 1032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:23.309275 1032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:23.309449 1032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:23.309511 1032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:23.342439 1032 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:39219
I20260928 15:58:23.342478 1160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:39219 every 8 connection(s)
I20260928 15:58:23.343726 1032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:58:23.347375 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 1032
I20260928 15:58:23.347590 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:58:23.348446 23031 external_mini_cluster.cc:1428] Setting key 50d95a6d83c876ed4174e74df5b5519b
Sep 28 15:58:23 dist-test-slave-9cqd krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611103, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:58:23.365772 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:23.346294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:34217 (local address 127.22.125.254:44759)
0928 15:58:23.346772 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:23.346780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:23.347888 (+ 1108us) server_negotiation.cc:407] Connection header received
0928 15:58:23.350022 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:23.350027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:23.350108 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:23.350250 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:23.352310 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.353204 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.355585 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.355979 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.356142 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:23.360184 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:23.360190 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:23.360193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:23.360224 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:23.362736 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:23.363434 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:23.363441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:23.363449 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:23.363558 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:23.364012 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:23.364017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:23.364020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:23.364349 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:23.364524 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:23.365178 (+ 654us) server_negotiation.cc:299] Negotiation successful
0928 15:58:23.365412 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":234,"threads_started":1}
I20260928 15:58:23.366739 1164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:23.346846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44759 (local address 127.22.125.193:34217)
0928 15:58:23.347696 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:23.347756 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:23.349760 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:23.350515 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:23.350526 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:23.351288 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:23.352125 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:23.352153 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.353356 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.353364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:23.355436 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:23.355453 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.356239 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.356249 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:23.356569 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:23.357313 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:23.357351 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:23.359924 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:23.362898 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:58:23.362906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:58:23.362921 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:58:23.363306 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:58:23.363667 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:58:23.363677 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:58:23.363680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:58:23.363890 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:58:23.364479 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:23.364488 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:23.364818 (+ 330us) client_negotiation.cc:769] Sending connection context
0928 15:58:23.365086 (+ 268us) client_negotiation.cc:240] Negotiation successful
0928 15:58:23.365599 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":302,"threads_started":1}
I20260928 15:58:23.368551 1161 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44759
I20260928 15:58:23.368850 1161 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:23.369580 1161 heartbeater.cc:511] Master 127.22.125.254:44759 requested a full tablet report, sending...
I20260928 15:58:23.371977 969 ts_manager.cc:194] Registered new tserver with Master: dc55c8d864b04f7ab9ca69f0e02bbc99 (127.22.125.193:39219)
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:58:23.373620 969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:34217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:58:23.400960 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44759
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:58:23.717758 1167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:23.718027 1167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:23.718149 1167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:23.725262 1167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:23.725392 1167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:23.725502 1167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:58:23.734968 1167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:23.736241 1167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:23.737488 1167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:23.750627 1175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:23.750682 1173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:23.751600 1167 server_base.cc:1034] running on GCE node
W20260928 15:58:23.752315 1172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:23.752940 1167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:23.753897 1167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:23.755137 1167 hybrid_clock.cc:648] HybridClock initialized: now 1790611103755084 us; error 46 us; skew 500 ppm
I20260928 15:58:23.755460 1167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:58:23 dist-test-slave-9cqd krb5kdc[924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611103, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:58:23.758776 1167 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:58:23.760406 1167 webserver.cc:460] Webserver started at http://127.22.125.194:33903/ using document root <none> and password file <none>
I20260928 15:58:23.761051 1167 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:23.761191 1167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:23.761497 1167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:23.764338 1167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "f3336e1c5ebc44df92c3126deb636bab"
format_stamp: "Formatted at 2026-09-28 15:58:23 on dist-test-slave-9cqd"
server_key: "218f0b6730f7e29170101bd95ded8f93"
server_key_iv: "3626d908100d76ca5c9e361ea08abb6f"
server_key_version: "encryptionkey@0"
I20260928 15:58:23.765082 1167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "f3336e1c5ebc44df92c3126deb636bab"
format_stamp: "Formatted at 2026-09-28 15:58:23 on dist-test-slave-9cqd"
server_key: "218f0b6730f7e29170101bd95ded8f93"
server_key_iv: "3626d908100d76ca5c9e361ea08abb6f"
server_key_version: "encryptionkey@0"
I20260928 15:58:23.769917 1167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 15:58:23.773236 1182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:23.774447 1167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 15:58:23.774622 1167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "f3336e1c5ebc44df92c3126deb636bab"
format_stamp: "Formatted at 2026-09-28 15:58:23 on dist-test-slave-9cqd"
server_key: "218f0b6730f7e29170101bd95ded8f93"
server_key_iv: "3626d908100d76ca5c9e361ea08abb6f"
server_key_version: "encryptionkey@0"
I20260928 15:58:23.774832 1167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:23.794896 1167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:23.799113 1167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:23.799433 1167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:23.800329 1167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:23.801649 1167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:23.801764 1167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:23.801910 1167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:23.801980 1167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:23.834982 1167 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:37209
I20260928 15:58:23.835035 1295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:37209 every 8 connection(s)
I20260928 15:58:23.836289 1167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:58:23.840620 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 1167
I20260928 15:58:23.840821 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:58:23.841320 23031 external_mini_cluster.cc:1428] Setting key 0ba5214d1addc8bb5a3a31f377c7a5b9
Sep 28 15:58:23 dist-test-slave-9cqd krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611103, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:58:23.856598 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:23.838984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:41959 (local address 127.22.125.254:44759)
0928 15:58:23.839246 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:23.839253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:23.840355 (+ 1102us) server_negotiation.cc:407] Connection header received
0928 15:58:23.842298 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:23.842303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:23.842398 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:23.842509 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:23.843916 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.844697 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.846211 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.846619 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.846755 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:23.850466 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:23.850471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:23.850474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:23.850505 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:23.853204 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:23.854141 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:23.854150 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:23.854155 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:23.854302 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:23.854844 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:23.854848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:23.854850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:23.855150 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:23.855270 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:23.856191 (+ 921us) server_negotiation.cc:299] Negotiation successful
0928 15:58:23.856364 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260928 15:58:23.858057 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:23.839451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44759 (local address 127.22.125.194:41959)
0928 15:58:23.840216 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:23.840253 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:23.842039 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:23.842739 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:23.842747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:23.843227 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:23.843751 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:23.843767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.844847 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.844857 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:23.846023 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:23.846033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:23.846839 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:23.846847 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:23.847074 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:23.847786 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:23.847813 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:23.850214 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:23.853338 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:58:23.853350 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:58:23.853367 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:58:23.854010 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:58:23.854420 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:58:23.854428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:58:23.854434 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:58:23.854698 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:58:23.855283 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:23.855293 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:23.855739 (+ 446us) client_negotiation.cc:769] Sending connection context
0928 15:58:23.856070 (+ 331us) client_negotiation.cc:240] Negotiation successful
0928 15:58:23.856620 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":271,"threads_started":1}
I20260928 15:58:23.859560 1296 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44759
I20260928 15:58:23.859889 1296 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:23.860838 1296 heartbeater.cc:511] Master 127.22.125.254:44759 requested a full tablet report, sending...
I20260928 15:58:23.862396 969 ts_manager.cc:194] Registered new tserver with Master: f3336e1c5ebc44df92c3126deb636bab (127.22.125.194:37209)
I20260928 15:58:23.863184 969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:41959
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:58:23.891490 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44759
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:58:24.201660 1301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:58:24.201972 1301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:58:24.202147 1301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:58:24.209101 1301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:58:24.209242 1301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:58:24.209429 1301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:58:24.219079 1301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:44759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:58:24.220359 1301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:58:24.221665 1301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:58:24.234454 1309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:24.234419 1307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:58:24.234488 1306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:58:24.234941 1301 server_base.cc:1034] running on GCE node
I20260928 15:58:24.235581 1301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:58:24.236485 1301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:58:24.237674 1301 hybrid_clock.cc:648] HybridClock initialized: now 1790611104237622 us; error 54 us; skew 500 ppm
I20260928 15:58:24.237973 1301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:58:24 dist-test-slave-9cqd krb5kdc[924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:58:24.241021 1301 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:58:24.242414 1301 webserver.cc:460] Webserver started at http://127.22.125.195:35747/ using document root <none> and password file <none>
I20260928 15:58:24.243059 1301 fs_manager.cc:359] Metadata directory not provided
I20260928 15:58:24.243207 1301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:58:24.243510 1301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 15:58:24.246367 1301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "1d1cef64fbb94563b36186b12321b4e8"
format_stamp: "Formatted at 2026-09-28 15:58:24 on dist-test-slave-9cqd"
server_key: "dbcab8aa6f7179f1b21a4a40a2aae704"
server_key_iv: "c3c8dc3d0594bd6b6c115e4fe96cbc07"
server_key_version: "encryptionkey@0"
I20260928 15:58:24.247121 1301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "1d1cef64fbb94563b36186b12321b4e8"
format_stamp: "Formatted at 2026-09-28 15:58:24 on dist-test-slave-9cqd"
server_key: "dbcab8aa6f7179f1b21a4a40a2aae704"
server_key_iv: "c3c8dc3d0594bd6b6c115e4fe96cbc07"
server_key_version: "encryptionkey@0"
I20260928 15:58:24.251999 1301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260928 15:58:24.255196 1316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:24.256515 1301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 15:58:24.256680 1301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "1d1cef64fbb94563b36186b12321b4e8"
format_stamp: "Formatted at 2026-09-28 15:58:24 on dist-test-slave-9cqd"
server_key: "dbcab8aa6f7179f1b21a4a40a2aae704"
server_key_iv: "c3c8dc3d0594bd6b6c115e4fe96cbc07"
server_key_version: "encryptionkey@0"
I20260928 15:58:24.256862 1301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:58:24.285830 1301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:58:24.289758 1301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:58:24.290055 1301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:58:24.290952 1301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:58:24.292228 1301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:58:24.292332 1301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:24.292483 1301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:58:24.292562 1301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:58:24.324985 1301 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:34825
I20260928 15:58:24.325031 1429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:34825 every 8 connection(s)
I20260928 15:58:24.326264 1301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:58:24.330633 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 1301
I20260928 15:58:24.330806 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:58:24.331372 23031 external_mini_cluster.cc:1428] Setting key f1e09280455b53db9830606a8880cd2e
Sep 28 15:58:24 dist-test-slave-9cqd krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:58:24.347244 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.328787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:42903 (local address 127.22.125.254:44759)
0928 15:58:24.328983 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.328987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.330080 (+ 1093us) server_negotiation.cc:407] Connection header received
0928 15:58:24.332609 (+ 2529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.332614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.332703 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.332815 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:24.334245 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.335011 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.336616 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.337030 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.337161 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.340428 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:24.340433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:24.340447 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:24.340477 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:24.343673 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.344440 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.344446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.344450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.344589 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.345140 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.345145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.345148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.345518 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:24.345684 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.346692 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.346920 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 15:58:24.348112 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.329262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:44759 (local address 127.22.125.195:42903)
0928 15:58:24.329928 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:24.329969 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:24.332293 (+ 2324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:24.333050 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:24.333058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:24.333547 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:58:24.334080 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.334096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.335160 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.335165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.336475 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.336487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.337258 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.337265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.337502 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.338186 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:58:24.338208 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:58:24.340256 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:58:24.343813 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:58:24.343819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:58:24.343829 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:58:24.344288 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:58:24.344744 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:58:24.344749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:58:24.344752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:58:24.344965 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:58:24.345761 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:58:24.345768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:58:24.346173 (+ 405us) client_negotiation.cc:769] Sending connection context
0928 15:58:24.346483 (+ 310us) client_negotiation.cc:240] Negotiation successful
0928 15:58:24.346968 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":258,"threads_started":1}
I20260928 15:58:24.349306 1430 heartbeater.cc:347] Connected to a master server at 127.22.125.254:44759
I20260928 15:58:24.349592 1430 heartbeater.cc:464] Registering TS with master...
I20260928 15:58:24.350258 1430 heartbeater.cc:511] Master 127.22.125.254:44759 requested a full tablet report, sending...
I20260928 15:58:24.351548 969 ts_manager.cc:194] Registered new tserver with Master: 1d1cef64fbb94563b36186b12321b4e8 (127.22.125.195:34825)
I20260928 15:58:24.352404 969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:42903
I20260928 15:58:24.358371 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:58:24.374318 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.362999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.22.125.254:44759)
0928 15:58:24.363160 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.363165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.363686 (+ 521us) server_negotiation.cc:407] Connection header received
0928 15:58:24.363817 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.363821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.363877 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.363954 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:24.365271 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.365972 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.367097 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.367497 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.367584 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.369640 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:24.369644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:24.369647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:24.369675 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:24.372236 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.372776 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.372779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.372781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.372901 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.373196 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.373200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.373201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.373492 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:24.373671 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.373966 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.374113 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 15:58:24.376297 1161 heartbeater.cc:503] Master 127.22.125.254:44759 was elected leader, sending a full tablet report...
I20260928 15:58:24.378530 968 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:58:24.385442 968 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:58:24.417727 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.397719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.22.125.193:39219)
0928 15:58:24.408978 (+ 11259us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.408992 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.409837 (+ 845us) server_negotiation.cc:407] Connection header received
0928 15:58:24.410110 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.410120 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.410437 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.411164 (+ 727us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:58:24.411460 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.412762 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.415043 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.416599 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.416803 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.416994 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.417234 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.417392 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11053,"thread_start_us":10896,"threads_started":1}
I20260928 15:58:24.417804 1443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.397454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39219 (local address 127.0.0.1:46450)
0928 15:58:24.409719 (+ 12265us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:24.409737 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:24.409886 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:24.410693 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:24.410703 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:24.410741 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:58:24.411011 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.411019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.412887 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.412893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.414872 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.414887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.416798 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.416804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.416984 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.417039 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 15:58:24.417192 (+ 153us) client_negotiation.cc:240] Negotiation successful
0928 15:58:24.417378 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12135,"mutex_wait_us":37,"thread_start_us":11962,"threads_started":1}
I20260928 15:58:24.418505 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.399894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:37209 (local address 127.0.0.1:37976)
0928 15:58:24.410450 (+ 10556us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:24.410469 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:24.410589 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:24.411283 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:24.411288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:24.411312 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:58:24.411888 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.411901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.413808 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.413814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.415661 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.415672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.417658 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.417668 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.417837 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.417869 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 15:58:24.417961 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 15:58:24.418093 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10413,"mutex_wait_us":23,"thread_start_us":10282,"threads_started":1}
I20260928 15:58:24.419073 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.400477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37976 (local address 127.22.125.194:37209)
0928 15:58:24.409433 (+ 8956us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.409443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.410716 (+ 1273us) server_negotiation.cc:407] Connection header received
0928 15:58:24.410822 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.410831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.411075 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.411274 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:58:24.412081 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.413660 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.415801 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.417205 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.417362 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.417516 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.418511 (+ 995us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.418751 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8779,"thread_start_us":8659,"threads_started":1}
I20260928 15:58:24.419389 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.397945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48108 (local address 127.22.125.195:34825)
0928 15:58:24.410810 (+ 12865us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.410818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.410844 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:58:24.410929 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.410935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.411104 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.411943 (+ 839us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:58:24.412273 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.414003 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.416591 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.418029 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.418188 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.418351 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.418900 (+ 549us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.419041 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12701,"thread_start_us":12564,"threads_started":1}
I20260928 15:58:24.420358 1442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.397454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:34825 (local address 127.0.0.1:48108)
0928 15:58:24.398102 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:24.398152 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:24.398321 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:24.411300 (+ 12979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:24.411304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:24.411330 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:58:24.411758 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.411768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.414151 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.414157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.416438 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.416453 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.418517 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.418525 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.418718 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.418750 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 15:58:24.418860 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 15:58:24.418972 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":738,"threads_started":1}
I20260928 15:58:24.422252 1096 tablet_service.cc:1467] Processing CreateTablet for tablet e2dca7cf2436493dbb7c0ee3d187e0a5 (DEFAULT_TABLE table=test-table [id=129c1f5737b8484996b47bd38052ab02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:58:24.422593 1231 tablet_service.cc:1467] Processing CreateTablet for tablet e2dca7cf2436493dbb7c0ee3d187e0a5 (DEFAULT_TABLE table=test-table [id=129c1f5737b8484996b47bd38052ab02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:58:24.423236 1365 tablet_service.cc:1467] Processing CreateTablet for tablet e2dca7cf2436493dbb7c0ee3d187e0a5 (DEFAULT_TABLE table=test-table [id=129c1f5737b8484996b47bd38052ab02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:58:24.423784 1231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2dca7cf2436493dbb7c0ee3d187e0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:58:24.423777 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2dca7cf2436493dbb7c0ee3d187e0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:58:24.424461 1365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2dca7cf2436493dbb7c0ee3d187e0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:58:24.436025 1448 tablet_bootstrap.cc:492] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Bootstrap starting.
I20260928 15:58:24.439035 1448 tablet_bootstrap.cc:654] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Neither blocks nor log segments found. Creating new log.
I20260928 15:58:24.439821 1449 tablet_bootstrap.cc:492] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Bootstrap starting.
I20260928 15:58:24.440392 1448 log.cc:824] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Log is configured to *not* fsync() on all Append() calls
I20260928 15:58:24.440706 1450 tablet_bootstrap.cc:492] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Bootstrap starting.
I20260928 15:58:24.443496 1448 tablet_bootstrap.cc:492] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: No bootstrap required, opened a new log
I20260928 15:58:24.443750 1448 ts_tablet_manager.cc:1397] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260928 15:58:24.444129 1449 tablet_bootstrap.cc:654] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Neither blocks nor log segments found. Creating new log.
I20260928 15:58:24.445333 1449 log.cc:824] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Log is configured to *not* fsync() on all Append() calls
I20260928 15:58:24.446964 1450 tablet_bootstrap.cc:654] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Neither blocks nor log segments found. Creating new log.
I20260928 15:58:24.448004 1450 log.cc:824] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Log is configured to *not* fsync() on all Append() calls
I20260928 15:58:24.449177 1449 tablet_bootstrap.cc:492] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: No bootstrap required, opened a new log
I20260928 15:58:24.449438 1449 ts_tablet_manager.cc:1397] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260928 15:58:24.450373 1450 tablet_bootstrap.cc:492] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: No bootstrap required, opened a new log
I20260928 15:58:24.450618 1450 ts_tablet_manager.cc:1397] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.002s
I20260928 15:58:24.454239 1448 raft_consensus.cc:348] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.454648 1448 raft_consensus.cc:374] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:58:24.454743 1448 raft_consensus.cc:729] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1cef64fbb94563b36186b12321b4e8, State: Initialized, Role: FOLLOWER
I20260928 15:58:24.455421 1448 consensus_queue.cc:260] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.456519 1430 heartbeater.cc:503] Master 127.22.125.254:44759 was elected leader, sending a full tablet report...
I20260928 15:58:24.457103 1448 ts_tablet_manager.cc:1428] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260928 15:58:24.457849 1450 raft_consensus.cc:348] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.458233 1450 raft_consensus.cc:374] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:58:24.458334 1450 raft_consensus.cc:729] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3336e1c5ebc44df92c3126deb636bab, State: Initialized, Role: FOLLOWER
I20260928 15:58:24.458974 1450 consensus_queue.cc:260] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.460106 1296 heartbeater.cc:503] Master 127.22.125.254:44759 was elected leader, sending a full tablet report...
I20260928 15:58:24.460659 1450 ts_tablet_manager.cc:1428] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260928 15:58:24.461728 1449 raft_consensus.cc:348] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.462149 1449 raft_consensus.cc:374] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:58:24.462267 1449 raft_consensus.cc:729] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc55c8d864b04f7ab9ca69f0e02bbc99, State: Initialized, Role: FOLLOWER
I20260928 15:58:24.462834 1449 consensus_queue.cc:260] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.464432 1449 ts_tablet_manager.cc:1428] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260928 15:58:24.544446 1455 raft_consensus.cc:482] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:58:24.544714 1455 raft_consensus.cc:504] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.546367 1455 leader_election.cc:288] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1cef64fbb94563b36186b12321b4e8 (127.22.125.195:34825), dc55c8d864b04f7ab9ca69f0e02bbc99 (127.22.125.193:39219)
I20260928 15:58:24.554298 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.546780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:53229 (local address 127.22.125.195:34825)
0928 15:58:24.547003 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.547011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.547714 (+ 703us) server_negotiation.cc:407] Connection header received
0928 15:58:24.547932 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.547937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.548005 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.548128 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:58:24.548712 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.549904 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.552350 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.553433 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.553596 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.553677 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.553906 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.554048 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 15:58:24.554405 1457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.546762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:34825 (local address 127.22.125.194:53229)
0928 15:58:24.547600 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:24.547616 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:24.547786 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:24.548147 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:24.548151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:24.548177 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:58:24.548567 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.548577 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.550055 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.550060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.552181 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.552201 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.553630 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.553634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.553739 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.553763 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 15:58:24.553848 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 15:58:24.553936 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":5,"thread_start_us":482,"threads_started":1}
I20260928 15:58:24.554986 1458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.546767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39219 (local address 127.22.125.194:49967)
0928 15:58:24.548624 (+ 1857us) negotiation.cc:107] Waiting for socket to connect
0928 15:58:24.548638 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 15:58:24.548776 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:58:24.549154 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:58:24.549158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:58:24.549184 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:58:24.549535 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.549546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.550694 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.550699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.552605 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:58:24.552615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.554172 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.554177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:58:24.554382 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.554442 (+ 60us) client_negotiation.cc:769] Sending connection context
0928 15:58:24.554590 (+ 148us) client_negotiation.cc:240] Negotiation successful
0928 15:58:24.554726 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1736,"mutex_wait_us":101,"thread_start_us":1404,"threads_started":1}
I20260928 15:58:24.555341 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.546854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:49967 (local address 127.22.125.193:39219)
0928 15:58:24.547073 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.547080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.548729 (+ 1649us) server_negotiation.cc:407] Connection header received
0928 15:58:24.548931 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.548935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.549023 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.549118 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:58:24.549737 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.550555 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.552751 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.553961 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.554108 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.554174 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.554979 (+ 805us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.555117 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 15:58:24.555413 1385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2dca7cf2436493dbb7c0ee3d187e0a5" candidate_uuid: "f3336e1c5ebc44df92c3126deb636bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cef64fbb94563b36186b12321b4e8" is_pre_election: true
I20260928 15:58:24.555966 1385 raft_consensus.cc:2454] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3336e1c5ebc44df92c3126deb636bab in term 0.
I20260928 15:58:24.556370 1116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2dca7cf2436493dbb7c0ee3d187e0a5" candidate_uuid: "f3336e1c5ebc44df92c3126deb636bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" is_pre_election: true
I20260928 15:58:24.556687 1183 leader_election.cc:302] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1cef64fbb94563b36186b12321b4e8, f3336e1c5ebc44df92c3126deb636bab; no voters:
I20260928 15:58:24.556962 1116 raft_consensus.cc:2454] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3336e1c5ebc44df92c3126deb636bab in term 0.
I20260928 15:58:24.557085 1455 raft_consensus.cc:2781] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:58:24.557269 1455 raft_consensus.cc:482] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:58:24.557389 1455 raft_consensus.cc:3037] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:58:24.559610 1455 raft_consensus.cc:504] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.560420 1455 leader_election.cc:288] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [CANDIDATE]: Term 1 election: Requested vote from peers 1d1cef64fbb94563b36186b12321b4e8 (127.22.125.195:34825), dc55c8d864b04f7ab9ca69f0e02bbc99 (127.22.125.193:39219)
I20260928 15:58:24.560693 1385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2dca7cf2436493dbb7c0ee3d187e0a5" candidate_uuid: "f3336e1c5ebc44df92c3126deb636bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1cef64fbb94563b36186b12321b4e8"
I20260928 15:58:24.560890 1385 raft_consensus.cc:3037] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:58:24.560814 1116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2dca7cf2436493dbb7c0ee3d187e0a5" candidate_uuid: "f3336e1c5ebc44df92c3126deb636bab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99"
I20260928 15:58:24.561055 1116 raft_consensus.cc:3037] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:58:24.563158 1385 raft_consensus.cc:2454] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3336e1c5ebc44df92c3126deb636bab in term 1.
I20260928 15:58:24.563266 1116 raft_consensus.cc:2454] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3336e1c5ebc44df92c3126deb636bab in term 1.
I20260928 15:58:24.563526 1183 leader_election.cc:302] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1cef64fbb94563b36186b12321b4e8, f3336e1c5ebc44df92c3126deb636bab; no voters:
I20260928 15:58:24.563818 1455 raft_consensus.cc:2781] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:58:24.564285 1455 raft_consensus.cc:686] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 1 LEADER]: Becoming Leader. State: Replica: f3336e1c5ebc44df92c3126deb636bab, State: Running, Role: LEADER
I20260928 15:58:24.564756 1455 consensus_queue.cc:237] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } }
I20260928 15:58:24.569418 968 catalog_manager.cc:5495] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab reported cstate change: term changed from 0 to 1, leader changed from <none> to f3336e1c5ebc44df92c3126deb636bab (127.22.125.194). New cstate: current_term: 1 leader_uuid: "f3336e1c5ebc44df92c3126deb636bab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3336e1c5ebc44df92c3126deb636bab" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37209 } health_report { overall_health: HEALTHY } } }
W20260928 15:58:24.577858 1431 tablet.cc:2369] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:58:24.588320 1297 tablet.cc:2369] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:58:24.596280 1162 tablet.cc:2369] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:58:24.599079 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.593761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37978 (local address 127.22.125.194:37209)
0928 15:58:24.593964 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.593972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.594060 (+ 88us) server_negotiation.cc:407] Connection header received
0928 15:58:24.594300 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.594304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.594394 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.594546 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:58:24.594913 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.595649 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.597196 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.597587 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.597696 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.598100 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:58:24.598614 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:58:24.598722 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.598807 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.598881 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 15:58:24.607110 1116 raft_consensus.cc:1261] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 1 FOLLOWER]: Refusing update from remote peer f3336e1c5ebc44df92c3126deb636bab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:58:24.607131 1385 raft_consensus.cc:1261] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 1 FOLLOWER]: Refusing update from remote peer f3336e1c5ebc44df92c3126deb636bab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:58:24.608021 1459 consensus_queue.cc:1035] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1cef64fbb94563b36186b12321b4e8" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 34825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:58:24.608441 1455 consensus_queue.cc:1035] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc55c8d864b04f7ab9ca69f0e02bbc99" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:58:24.631228 969 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:58:24.631477 969 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:58:24.634649 969 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 291059f6fdeb42319a3158498d233118: Sending DeleteTablet for 3 replicas of tablet e2dca7cf2436493dbb7c0ee3d187e0a5
I20260928 15:58:24.635823 1365 tablet_service.cc:1514] Processing DeleteTablet for tablet e2dca7cf2436493dbb7c0ee3d187e0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:58:24 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:48108
I20260928 15:58:24.635815 1096 tablet_service.cc:1514] Processing DeleteTablet for tablet e2dca7cf2436493dbb7c0ee3d187e0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:58:24 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:46450
I20260928 15:58:24.635982 1231 tablet_service.cc:1514] Processing DeleteTablet for tablet e2dca7cf2436493dbb7c0ee3d187e0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:58:24 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:37976
I20260928 15:58:24.637012 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 1032
I20260928 15:58:24.648484 1472 tablet_replica.cc:330] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: stopping tablet replica
I20260928 15:58:24.649531 1472 raft_consensus.cc:2229] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:58:24.650611 1472 raft_consensus.cc:2258] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:58:24.651563 1474 tablet_replica.cc:330] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: stopping tablet replica
I20260928 15:58:24.651939 1474 raft_consensus.cc:2229] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:58:24.652586 1474 raft_consensus.cc:2258] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:58:24.653468 1472 ts_tablet_manager.cc:1905] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:58:24.654029 1473 tablet_replica.cc:330] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: stopping tablet replica
I20260928 15:58:24.654431 1473 raft_consensus.cc:2229] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:58:24.654873 1473 raft_consensus.cc:2258] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:58:24.656903 1473 ts_tablet_manager.cc:1905] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:58:24.657503 1474 ts_tablet_manager.cc:1905] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:58:24.661963 1472 ts_tablet_manager.cc:1918] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:58:24.662178 1472 log.cc:1196] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/e2dca7cf2436493dbb7c0ee3d187e0a5
I20260928 15:58:24.662772 1472 ts_tablet_manager.cc:1939] T e2dca7cf2436493dbb7c0ee3d187e0a5 P dc55c8d864b04f7ab9ca69f0e02bbc99: Deleting consensus metadata
I20260928 15:58:24.663820 1473 ts_tablet_manager.cc:1918] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:58:24.663964 1473 log.cc:1196] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/e2dca7cf2436493dbb7c0ee3d187e0a5
I20260928 15:58:24.664465 1473 ts_tablet_manager.cc:1939] T e2dca7cf2436493dbb7c0ee3d187e0a5 P 1d1cef64fbb94563b36186b12321b4e8: Deleting consensus metadata
I20260928 15:58:24.664861 954 catalog_manager.cc:4841] TS dc55c8d864b04f7ab9ca69f0e02bbc99 (127.22.125.193:39219): tablet e2dca7cf2436493dbb7c0ee3d187e0a5 (table test-table [id=129c1f5737b8484996b47bd38052ab02]) successfully deleted
I20260928 15:58:24.665434 1474 ts_tablet_manager.cc:1918] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:58:24.665596 1474 log.cc:1196] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/e2dca7cf2436493dbb7c0ee3d187e0a5
I20260928 15:58:24.666203 1474 ts_tablet_manager.cc:1939] T e2dca7cf2436493dbb7c0ee3d187e0a5 P f3336e1c5ebc44df92c3126deb636bab: Deleting consensus metadata
I20260928 15:58:24.666205 953 catalog_manager.cc:4841] TS 1d1cef64fbb94563b36186b12321b4e8 (127.22.125.195:34825): tablet e2dca7cf2436493dbb7c0ee3d187e0a5 (table test-table [id=129c1f5737b8484996b47bd38052ab02]) successfully deleted
I20260928 15:58:24.667829 953 catalog_manager.cc:4841] TS f3336e1c5ebc44df92c3126deb636bab (127.22.125.194:37209): tablet e2dca7cf2436493dbb7c0ee3d187e0a5 (table test-table [id=129c1f5737b8484996b47bd38052ab02]) successfully deleted
Sep 28 15:58:24 dist-test-slave-9cqd krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:58:24.677372 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.638604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46466 (local address 127.22.125.193:39219)
0928 15:58:24.638818 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.638827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.648586 (+ 9759us) server_negotiation.cc:407] Connection header received
0928 15:58:24.649117 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.649126 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.649205 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.649269 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:24.652969 (+ 3700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.654095 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.664770 (+ 10675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.665416 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.665529 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.670803 (+ 5274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:24.670817 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:24.670835 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:24.670883 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:24.674551 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.675196 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.675206 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.675210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.675379 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.675747 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.675752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.675756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.676291 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:24.676521 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.676891 (+ 370us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.677047 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 15:58:24.678092 1156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:24.778157 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 1032
I20260928 15:58:24.796645 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 1167
Sep 28 15:58:24 dist-test-slave-9cqd krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:58:24.814306 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.797566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.22.125.194:37209)
0928 15:58:24.797806 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.797814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.798070 (+ 256us) server_negotiation.cc:407] Connection header received
0928 15:58:24.798301 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.798304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.798379 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.798462 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:24.800138 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.800932 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.802737 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.803274 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.803367 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.808281 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:24.808299 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:24.808314 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:24.808356 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:24.811315 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.811991 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.811998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.812000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.812379 (+ 379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.812756 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.812762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.812765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.813124 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:24.813494 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.813716 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.813954 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 15:58:24.814975 1291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:24.932181 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 1167
I20260928 15:58:24.950263 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 1301
Sep 28 15:58:24 dist-test-slave-9cqd krb5kdc[924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:58:24.964712 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:58:24.950929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.22.125.195:34825)
0928 15:58:24.951086 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 15:58:24.951090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:58:24.951108 (+ 18us) server_negotiation.cc:407] Connection header received
0928 15:58:24.951268 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:58:24.951271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:58:24.951344 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:58:24.951445 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:58:24.952742 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.953450 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.954641 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:58:24.955019 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:58:24.955106 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:58:24.959438 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:58:24.959450 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:58:24.959466 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:58:24.959497 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:58:24.962301 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.962890 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.962898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.962900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.963037 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:58:24.963326 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:58:24.963329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:58:24.963331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:58:24.963650 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:58:24.963887 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0928 15:58:24.964206 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 15:58:24.964407 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 15:58:24.965438 1425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:25.053175 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 1301
I20260928 15:58:25.069479 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 937
I20260928 15:58:25.070564 1000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:58:25.179560 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 937
2026-09-28T15:58:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4192 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:58:25 dist-test-slave-9cqd krb5kdc[1482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:58:25 dist-test-slave-9cqd krb5kdc[1482](info): set up 2 sockets
Sep 28 15:58:25 dist-test-slave-9cqd krb5kdc[1482](info): commencing operation
krb5kdc: starting...
W20260928 15:58:26.290858 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:58:26 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:58:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:58:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/rangeradmin_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/rangeradmin_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/rangerlookup_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/rangerlookup_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
I20260928 15:58:26.593245 23031 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYsP_Ep/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/postgres -l logfile start
2026-09-28 15:58:29.114 UTC [1512] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 15:58:29.114 UTC [1512] LOG: listening on IPv4 address "127.22.125.212", port 47575
2026-09-28 15:58:29.117 UTC [1512] LOG: listening on Unix socket "/tmp/.s.PGSQL.47575"
2026-09-28 15:58:29.139 UTC [1515] LOG: database system was shut down at 2026-09-28 15:58:27 UTC
2026-09-28 15:58:29.142 UTC [1512] LOG: database system is ready to accept connections
I20260928 15:58:30.110435 23031 mini_postgres.cc:96] Postgres bound to 47575
2026-09-28 15:58:30.128 UTC [1523] FATAL: database "slave" does not exist
127.22.125.212:47575 - accepting connections
I20260928 15:58:30.129611 23031 mini_ranger.cc:161] Starting Ranger...
I20260928 15:58:30.156987 23031 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 15:58:30.621770 23031 mini_ranger.cc:87] Created ranger Postgres database
I20260928 15:58:30.621937 23031 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-admin
2026-09-28 15:58:31,104 [I] DB FLAVOR :POSTGRES
2026-09-28 15:58:31,104 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 15:58:31,104 [I] Checking connection..
2026-09-28 15:58:31,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 15:58:31,392 [I] Checking connection passed.
2026-09-28 15:58:31,392 [I] --------- Verifying version history table ---------
2026-09-28 15:58:31,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 15:58:31,630 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 15:58:31,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 15:58:31,889 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 15:58:31,889 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 15:58:31,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 15:58:32.135 UTC [1587] WARNING: there is no transaction in progress
2026-09-28 15:58:32,142 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 15:58:32,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 15:58:32,377 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 15:58:32,377 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 15:58:32,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 15:58:32,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 15:58:32,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9cqd.c.gcp-upstream.internal','N') ;"
2026-09-28 15:58:33,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 15:58:33,357 [I] Table x_portal_user does not exist in database ranger
2026-09-28 15:58:33,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 15:58:33,614 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 15:58:33,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 15:58:33,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 15:58:34,093 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 15:58:34,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 15:58:35.016 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.031 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.047 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.063 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.078 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.095 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.148 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.155 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.169 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.179 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.188 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.202 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.211 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.217 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.224 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35.230 UTC [1697] WARNING: there is no transaction in progress
2026-09-28 15:58:35,829 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 15:58:35,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 15:58:36,080 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 15:58:36,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 15:58:36,320 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 15:58:36,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 15:58:36,566 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 15:58:36,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 15:58:36,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 15:58:37,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9cqd.c.gcp-upstream.internal';"
2026-09-28 15:58:37,282 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 15:58:37,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 15:58:37,518 [I] DB_PATCHES have already been applied
I20260928 15:58:37.523906 23031 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYsP_Ep/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 15:58:37.524015 23031 mini_ranger.cc:193] Using host: 127.22.125.212
I20260928 15:58:37.538874 23031 mini_ranger.cc:227] Ranger admin URL: http://127.22.125.212:59149
Sep 28, 2026 3:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 3:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 3:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 3:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 3:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 3:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 3:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 15:58:39.834800 23031 mini_ranger.cc:160] Time spent starting Ranger: real 9.705s user 0.004s sys 0.040s
Sep 28, 2026 3:58:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 3:58:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 3:58:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 3:58:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 3:58:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 15:59:00.204367 23031 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/rangerkms_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/rangerkms_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.125.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.125.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 15:59:00 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611140, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 15:59:00.383428 23031 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 15:59:00.403553 23031 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 15:59:01.251992 23031 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 15:59:01.252166 23031 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms
I20260928 15:59:01.252321 23031 mini_ranger_kms.cc:227] Using postgres at 127.22.125.212:47575
2026-09-28 15:59:01,962 [I] DB FLAVOR :POSTGRES
2026-09-28 15:59:01,963 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 15:59:01,963 [I] Checking connection
2026-09-28 15:59:01,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 15:59:02,222 [I] connection success
2026-09-28 15:59:02,222 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 15:59:02,222 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 15:59:02,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 15:59:02,444 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 15:59:02,444 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 15:59:02,444 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 15:59:02,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:47575/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 15:59:02,711 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 15:59:03.190246 23031 mini_ranger_kms.cc:310] Created kms service
I20260928 15:59:03.356825 23031 mini_ranger_kms.cc:326] Created kudu user
I20260928 15:59:03.401459 23031 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 15:59:03 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611140, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:03.840408 23031 mini_ranger_kms.cc:384] Added ranger policy
I20260928 15:59:03.840703 23031 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 15:59:03.840794 23031 mini_ranger_kms.cc:240] Using host: 127.22.125.212
I20260928 15:59:03.858739 23031 mini_ranger_kms.cc:276] Ranger KMS PID: 1879
I20260928 15:59:03.858963 23031 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.125.212:59337
Sep 28, 2026 3:59:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 3:59:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 3:59:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 3:59:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 3:59:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 3:59:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 28 15:59:06 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790611146, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.125.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:06.152362 23031 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.769s user 0.128s sys 0.381s
I20260928 15:59:06.152616 23031 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 15:59:06.152853 23031 mini_ranger_kms.cc:401] 127.22.125.212:59337/kms/v1/keys
Sep 28 15:59:06 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611146, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.125.212@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
Sep 28 15:59:06 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611146, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.125.212@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
Sep 28 15:59:09 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:59:09.246081 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35703
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:35703
--ranger_config_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:59337/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:59:09.571431 1930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:59:09.571727 1930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:59:09.571892 1930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:59:09.579025 1930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:59:09.579123 1930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:59:09.579229 1930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:59:09.579324 1930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:59:09.579409 1930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:59:09.589340 1930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.125.212:59337/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:35703
--ranger_config_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:59:09.590553 1930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:59:09.591856 1930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:59:09.604326 1935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:09.604250 1938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:09.604295 1936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:59:09.604919 1930 server_base.cc:1034] running on GCE node
I20260928 15:59:09.605656 1930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:59:09.607023 1930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:59:09.608242 1930 hybrid_clock.cc:648] HybridClock initialized: now 1790611149608201 us; error 47 us; skew 500 ppm
I20260928 15:59:09.608603 1930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:59:09 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611149, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:09.615401 1930 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:59:09.616837 1930 webserver.cc:460] Webserver started at http://127.22.125.254:46781/ using document root <none> and password file <none>
I20260928 15:59:09.617489 1930 fs_manager.cc:359] Metadata directory not provided
I20260928 15:59:09.617624 1930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:59:09.617991 1930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 15:59:09 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611149, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:09.683333 1930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3"
format_stamp: "Formatted at 2026-09-28 15:59:09 on dist-test-slave-9cqd"
server_key: "4261d4acc759eb3dbf80b449691ecff3"
server_key_iv: "16ba53d6f7589126a91461de988c23a6"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:09.684319 1930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3"
format_stamp: "Formatted at 2026-09-28 15:59:09 on dist-test-slave-9cqd"
server_key: "4261d4acc759eb3dbf80b449691ecff3"
server_key_iv: "16ba53d6f7589126a91461de988c23a6"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:09.689942 1930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 15:59:09.709233 1945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:09.710964 1930 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 15:59:09.711169 1930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3"
format_stamp: "Formatted at 2026-09-28 15:59:09 on dist-test-slave-9cqd"
server_key: "4261d4acc759eb3dbf80b449691ecff3"
server_key_iv: "16ba53d6f7589126a91461de988c23a6"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:09.711452 1930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:59:09.731629 1930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:59:09.736179 1930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:59:09.736477 1930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:59:09.756286 1930 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:35703
I20260928 15:59:09.756290 1997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:35703 every 8 connection(s)
I20260928 15:59:09.758140 1930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:59:09.760290 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 1930
I20260928 15:59:09.760521 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:59:09.761320 23031 external_mini_cluster.cc:1428] Setting key 684bfe86ed73c11795aa9e634334e5d9
I20260928 15:59:09.762844 1998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:59:09.781313 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3: Bootstrap starting.
I20260928 15:59:09.785282 1998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3: Neither blocks nor log segments found. Creating new log.
I20260928 15:59:09.786466 1998 log.cc:824] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3: Log is configured to *not* fsync() on all Append() calls
I20260928 15:59:09.789728 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3: No bootstrap required, opened a new log
Sep 28 15:59:09 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:59:09.799620 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:09.763417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.22.125.254:35703)
0928 15:59:09.778154 (+ 14737us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:09.778170 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:09.778227 (+ 57us) server_negotiation.cc:407] Connection header received
0928 15:59:09.781004 (+ 2777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:09.781033 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:09.781577 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:09.782031 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:09.783570 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:09.785732 (+ 2162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:09.787595 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:09.788357 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:09.788559 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:09.792328 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:09.792341 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:09.792355 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:09.792394 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:09.795385 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:09.796074 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:09.796083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:09.796094 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:09.796263 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:09.796678 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:09.796685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:09.796688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:09.797329 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:09.797666 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:09.798359 (+ 693us) server_negotiation.cc:299] Negotiation successful
0928 15:59:09.798650 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14477,"thread_start_us":18036,"threads_started":1}
I20260928 15:59:09.800745 1998 raft_consensus.cc:348] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35703 } }
I20260928 15:59:09.801172 1998 raft_consensus.cc:374] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:59:09.801266 1998 raft_consensus.cc:729] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f55f43dcda4b47eaadf0fdc3c84b65d3, State: Initialized, Role: FOLLOWER
I20260928 15:59:09.801867 1998 consensus_queue.cc:260] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35703 } }
I20260928 15:59:09.802139 1998 raft_consensus.cc:388] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:59:09.802274 1998 raft_consensus.cc:482] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:59:09.802459 1998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:59:09.805672 1998 raft_consensus.cc:504] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35703 } }
I20260928 15:59:09.806227 1998 leader_election.cc:302] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f55f43dcda4b47eaadf0fdc3c84b65d3; no voters:
I20260928 15:59:09.806828 1998 leader_election.cc:288] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:59:09.806967 2003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:59:09.808291 2003 raft_consensus.cc:686] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [term 1 LEADER]: Becoming Leader. State: Replica: f55f43dcda4b47eaadf0fdc3c84b65d3, State: Running, Role: LEADER
I20260928 15:59:09.808993 1998 sys_catalog.cc:564] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:59:09.808964 2003 consensus_queue.cc:237] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35703 } }
I20260928 15:59:09.811950 2004 sys_catalog.cc:455] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35703 } } }
I20260928 15:59:09.812398 2004 sys_catalog.cc:458] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:59:09.811939 2005 sys_catalog.cc:455] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f55f43dcda4b47eaadf0fdc3c84b65d3. Latest consensus state: current_term: 1 leader_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f55f43dcda4b47eaadf0fdc3c84b65d3" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35703 } } }
I20260928 15:59:09.812963 2005 sys_catalog.cc:458] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3 [sys.catalog]: This master's current role is: LEADER
I20260928 15:59:09.814708 2009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:59:09.823323 2009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:59:09.824153 1998 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 15:59:09.839504 2009 catalog_manager.cc:1347] Generated new cluster ID: 8521944629804302a54216f2c1608662
I20260928 15:59:09.839888 2009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:59:09.867735 2009 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:59:09.869774 2009 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:59:09.883616 2009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3: Generated new TSK 0
I20260928 15:59:09.884725 2009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:59:10.603823 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:59337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35703
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 15:59:10 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790611150, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 15:59:11.003818 2043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:59:11.004206 2043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:59:11.004380 2043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:59:11.016006 2043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:59:11.016179 2043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:59:11.016384 2043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:59:11.031798 2043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.125.212:59337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:59:11.033610 2043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:59:11.035405 2043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:59:11.053861 2053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:11.054770 2054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:11.057006 2056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:59:11.057379 2043 server_base.cc:1034] running on GCE node
I20260928 15:59:11.058048 2043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:59:11.059059 2043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:59:11.060300 2043 hybrid_clock.cc:648] HybridClock initialized: now 1790611151060235 us; error 65 us; skew 500 ppm
I20260928 15:59:11.060683 2043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:59:11 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611151, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:11.064406 2043 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:59:11.066133 2043 webserver.cc:460] Webserver started at http://127.22.125.193:33199/ using document root <none> and password file <none>
I20260928 15:59:11.066845 2043 fs_manager.cc:359] Metadata directory not provided
I20260928 15:59:11.066989 2043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:59:11.067297 2043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 15:59:11 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611151, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:11.084677 2043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "f3f061e79d464d8e8ca242f991b36a56"
format_stamp: "Formatted at 2026-09-28 15:59:11 on dist-test-slave-9cqd"
server_key: "8e9f97747b0a04127769ec3e9fecd448"
server_key_iv: "286e23989246706497b506dbd76c805e"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:11.085834 2043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "f3f061e79d464d8e8ca242f991b36a56"
format_stamp: "Formatted at 2026-09-28 15:59:11 on dist-test-slave-9cqd"
server_key: "8e9f97747b0a04127769ec3e9fecd448"
server_key_iv: "286e23989246706497b506dbd76c805e"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:11.110250 2043 fs_manager.cc:691] Time spent creating directory manager: real 0.024s user 0.011s sys 0.012s
I20260928 15:59:11.130031 2063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:11.131951 2043 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 15:59:11.132153 2043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "f3f061e79d464d8e8ca242f991b36a56"
format_stamp: "Formatted at 2026-09-28 15:59:11 on dist-test-slave-9cqd"
server_key: "8e9f97747b0a04127769ec3e9fecd448"
server_key_iv: "286e23989246706497b506dbd76c805e"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:11.132445 2043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:59:11.178723 2043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:59:11.184192 2043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:59:11.184631 2043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:59:11.185963 2043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:59:11.187844 2043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:59:11.187951 2043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:11.188100 2043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:59:11.188189 2043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:11.259635 2043 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:39755
I20260928 15:59:11.259686 2175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:39755 every 8 connection(s)
I20260928 15:59:11.261476 2043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:59:11.263542 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 2043
I20260928 15:59:11.263752 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:59:11.264667 23031 external_mini_cluster.cc:1428] Setting key a4b5bd5e51202e385d43c614b5c6fe62
Sep 28 15:59:11 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611151, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:59:11.306424 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:11.271852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:40567 (local address 127.22.125.254:35703)
0928 15:59:11.284864 (+ 13012us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:11.284871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:11.284898 (+ 27us) server_negotiation.cc:407] Connection header received
0928 15:59:11.287394 (+ 2496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:11.287401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:11.287512 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:11.287642 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:11.289949 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:11.291121 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:11.293607 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:11.294253 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:11.294363 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:11.298984 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:11.298991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:11.298996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:11.299047 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:11.302667 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:11.303601 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:11.303607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:11.303611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:11.303754 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:11.304299 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:11.304304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:11.304308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:11.304681 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:11.304813 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:11.305755 (+ 942us) server_negotiation.cc:299] Negotiation successful
0928 15:59:11.306014 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12864,"thread_start_us":13043,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:59:11.308828 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:11.271231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35703 (local address 127.22.125.193:40567)
0928 15:59:11.278476 (+ 7245us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:11.278532 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:11.286997 (+ 8465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:11.288188 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:11.288202 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:11.288886 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:59:11.289735 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:11.289761 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:11.291353 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:11.291358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:11.293437 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:11.293452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:11.294726 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:11.294735 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:11.295040 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:11.295895 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:59:11.295927 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:59:11.298716 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:59:11.302855 (+ 4139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:11.302864 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:11.302883 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:11.303461 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:11.303897 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:11.303902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:11.303906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:11.304167 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:11.304908 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:59:11.304917 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:59:11.305329 (+ 412us) client_negotiation.cc:769] Sending connection context
0928 15:59:11.306702 (+ 1373us) client_negotiation.cc:240] Negotiation successful
0928 15:59:11.307248 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6916,"thread_start_us":6343,"threads_started":1}
I20260928 15:59:11.312189 2176 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35703
I20260928 15:59:11.312549 2176 heartbeater.cc:464] Registering TS with master...
I20260928 15:59:11.313426 2176 heartbeater.cc:511] Master 127.22.125.254:35703 requested a full tablet report, sending...
I20260928 15:59:11.316087 1963 ts_manager.cc:194] Registered new tserver with Master: f3f061e79d464d8e8ca242f991b36a56 (127.22.125.193:39755)
I20260928 15:59:11.318222 1963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:40567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:59:11.344370 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:59337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35703
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 15:59:11 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611150, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
Sep 28 15:59:11 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611150, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
W20260928 15:59:11.808820 2182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:59:11.809288 2182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:59:11.809466 2182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:59:11.820484 2182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:59:11.820623 2182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:59:11.820833 2182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:59:11.835323 2182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.125.212:59337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:59:11.837236 2182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:59:11.839067 2182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:59:11.864382 2189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:11.865451 2190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:11.874014 2193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:59:11.875012 2182 server_base.cc:1034] running on GCE node
I20260928 15:59:11.875813 2182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:59:11.877915 2182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:59:11.879238 2182 hybrid_clock.cc:648] HybridClock initialized: now 1790611151879119 us; error 100 us; skew 500 ppm
I20260928 15:59:11.880138 2182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:59:11 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611151, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:11.884567 2182 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:59:11.903895 2182 webserver.cc:460] Webserver started at http://127.22.125.194:45869/ using document root <none> and password file <none>
I20260928 15:59:11.905059 2182 fs_manager.cc:359] Metadata directory not provided
I20260928 15:59:11.905288 2182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:59:11.905736 2182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 15:59:11 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611151, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:11.926468 2182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "d15e1bcab84948f9bcd1b1b4213c80ea"
format_stamp: "Formatted at 2026-09-28 15:59:11 on dist-test-slave-9cqd"
server_key: "6c136984c00692a70e3e8af9cfbe4e85"
server_key_iv: "a4493a3a4f9f828edc940e73fbbeed50"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:11.927557 2182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "d15e1bcab84948f9bcd1b1b4213c80ea"
format_stamp: "Formatted at 2026-09-28 15:59:11 on dist-test-slave-9cqd"
server_key: "6c136984c00692a70e3e8af9cfbe4e85"
server_key_iv: "a4493a3a4f9f828edc940e73fbbeed50"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:11.950212 2182 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.009s sys 0.011s
I20260928 15:59:11.986344 2199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:11.992607 2182 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.006s sys 0.012s
I20260928 15:59:11.992904 2182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "d15e1bcab84948f9bcd1b1b4213c80ea"
format_stamp: "Formatted at 2026-09-28 15:59:11 on dist-test-slave-9cqd"
server_key: "6c136984c00692a70e3e8af9cfbe4e85"
server_key_iv: "a4493a3a4f9f828edc940e73fbbeed50"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:11.993258 2182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:59:12.070209 2182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:59:12.076663 2182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:59:12.076984 2182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:59:12.078243 2182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:59:12.080066 2182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:59:12.080169 2182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260928 15:59:12.080288 2182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:59:12.080348 2182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:12.119045 2182 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:34967
I20260928 15:59:12.119087 2318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:34967 every 8 connection(s)
I20260928 15:59:12.120853 2182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 15:59:12.127987 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 2182
I20260928 15:59:12.128212 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 15:59:12.128722 23031 external_mini_cluster.cc:1428] Setting key 463943aeea2cb88d2414a0d3e59464af
Sep 28 15:59:12 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611151, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 15:59:12.168922 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.124334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:38161 (local address 127.22.125.254:35703)
0928 15:59:12.141948 (+ 17614us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:12.141957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:12.143145 (+ 1188us) server_negotiation.cc:407] Connection header received
0928 15:59:12.145591 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:12.145597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:12.145692 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:12.145800 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:12.147918 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.149125 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.156160 (+ 7035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.156791 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.156932 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.161563 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:12.161570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:12.161575 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:12.161616 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:12.165242 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:12.166228 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:12.166233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:12.166237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:12.166379 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:12.166877 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:12.166882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:12.166885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:12.167247 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:12.167384 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:12.168320 (+ 936us) server_negotiation.cc:299] Negotiation successful
0928 15:59:12.168544 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17445,"thread_start_us":17302,"threads_started":1}
I20260928 15:59:12.170256 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.124694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35703 (local address 127.22.125.194:38161)
0928 15:59:12.142609 (+ 17915us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:12.142663 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:12.145171 (+ 2508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:12.146284 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:12.146297 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:12.146929 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:59:12.147726 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.147750 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.149324 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.149332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.151341 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.151355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.157224 (+ 5869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.157231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.157594 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.158432 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:59:12.158463 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:59:12.161318 (+ 2855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:59:12.165436 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:12.165444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:12.165461 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:12.166055 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:12.166502 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:12.166507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:12.166511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:12.166744 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:12.167413 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:59:12.167421 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:59:12.167843 (+ 422us) client_negotiation.cc:769] Sending connection context
0928 15:59:12.168164 (+ 321us) client_negotiation.cc:240] Negotiation successful
0928 15:59:12.168677 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17595,"thread_start_us":17384,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
I20260928 15:59:12.171943 2319 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35703
I20260928 15:59:12.172328 2319 heartbeater.cc:464] Registering TS with master...
I20260928 15:59:12.173164 2319 heartbeater.cc:511] Master 127.22.125.254:35703 requested a full tablet report, sending...
I20260928 15:59:12.174793 1963 ts_manager.cc:194] Registered new tserver with Master: d15e1bcab84948f9bcd1b1b4213c80ea (127.22.125.194:34967)
I20260928 15:59:12.175848 1963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:38161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 15:59:12.203518 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:59337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35703
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 15:59:12.322265 2176 heartbeater.cc:503] Master 127.22.125.254:35703 was elected leader, sending a full tablet report...
W20260928 15:59:12.581368 2325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:59:12.581686 2325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:59:12.581841 2325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:59:12.588974 2325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:59:12.589110 2325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:59:12.589277 2325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 15:59:12.598501 2325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.125.212:59337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:59:12.599831 2325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:59:12.601182 2325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:59:12.614041 2330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:12.614073 2333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:12.614220 2331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:59:12.614748 2325 server_base.cc:1034] running on GCE node
I20260928 15:59:12.615337 2325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:59:12.616246 2325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:59:12.617466 2325 hybrid_clock.cc:648] HybridClock initialized: now 1790611152617410 us; error 47 us; skew 500 ppm
I20260928 15:59:12.617817 2325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:59:12 dist-test-slave-9cqd krb5kdc[1482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611152, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:12.620958 2325 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 15:59:12.622279 2325 webserver.cc:460] Webserver started at http://127.22.125.195:39015/ using document root <none> and password file <none>
I20260928 15:59:12.622926 2325 fs_manager.cc:359] Metadata directory not provided
I20260928 15:59:12.623065 2325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:59:12.623375 2325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 15:59:12 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611152, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:12.635895 2325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "9125ae29485b4363bbd85ceb7a8dd631"
format_stamp: "Formatted at 2026-09-28 15:59:12 on dist-test-slave-9cqd"
server_key: "e2c1ae20526b37d5c65052569b9cd22c"
server_key_iv: "fc36ce5a382ed2f7583835cfd0d50fb3"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:12.636969 2325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "9125ae29485b4363bbd85ceb7a8dd631"
format_stamp: "Formatted at 2026-09-28 15:59:12 on dist-test-slave-9cqd"
server_key: "e2c1ae20526b37d5c65052569b9cd22c"
server_key_iv: "fc36ce5a382ed2f7583835cfd0d50fb3"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:12.643983 2325 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260928 15:59:12.657748 2340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:12.659209 2325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 15:59:12.659391 2325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "9125ae29485b4363bbd85ceb7a8dd631"
format_stamp: "Formatted at 2026-09-28 15:59:12 on dist-test-slave-9cqd"
server_key: "e2c1ae20526b37d5c65052569b9cd22c"
server_key_iv: "fc36ce5a382ed2f7583835cfd0d50fb3"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:12.659606 2325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:59:12.673169 2325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:59:12.677143 2325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:59:12.677469 2325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:59:12.678378 2325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:59:12.679733 2325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:59:12.679847 2325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:12.679989 2325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:59:12.680075 2325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:12.713382 2325 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:36213
I20260928 15:59:12.713418 2453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:36213 every 8 connection(s)
I20260928 15:59:12.714627 2325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 15:59:12.723878 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 2325
I20260928 15:59:12.724107 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 15:59:12.724658 23031 external_mini_cluster.cc:1428] Setting key c8eb840a78411dffec7a787cb1b6f806
Sep 28 15:59:12 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611152, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:59:12.737135 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.717105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:39635 (local address 127.22.125.254:35703)
0928 15:59:12.717598 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:12.717603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:12.718400 (+ 797us) server_negotiation.cc:407] Connection header received
0928 15:59:12.720360 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:12.720366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:12.720457 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:12.720576 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:12.721945 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.722695 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.724304 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.724975 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.725120 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.728752 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:12.728759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:12.728764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:12.728831 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:12.733158 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:12.734270 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:12.734291 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:12.734296 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:12.734494 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:12.735106 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:12.735112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:12.735116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:12.735515 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:12.735673 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:12.736568 (+ 895us) server_negotiation.cc:299] Negotiation successful
0928 15:59:12.736764 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
I20260928 15:59:12.738314 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.717551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35703 (local address 127.22.125.195:39635)
0928 15:59:12.718236 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:12.718274 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:12.720032 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:12.720808 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:12.720817 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:12.721226 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:59:12.721778 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.721795 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.722871 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.722877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.724137 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.724148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.725204 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.725212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.725531 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.726281 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:59:12.726303 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:59:12.728485 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:59:12.733311 (+ 4826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:12.733320 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:12.733335 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:12.734142 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:12.734605 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:12.734610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:12.734614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:12.734923 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:12.735659 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:59:12.735666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:59:12.735986 (+ 320us) client_negotiation.cc:769] Sending connection context
0928 15:59:12.736465 (+ 479us) client_negotiation.cc:240] Negotiation successful
0928 15:59:12.736958 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":273,"threads_started":1}
I20260928 15:59:12.739938 2454 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35703
I20260928 15:59:12.740285 2454 heartbeater.cc:464] Registering TS with master...
I20260928 15:59:12.741097 2454 heartbeater.cc:511] Master 127.22.125.254:35703 requested a full tablet report, sending...
I20260928 15:59:12.742695 1963 ts_manager.cc:194] Registered new tserver with Master: 9125ae29485b4363bbd85ceb7a8dd631 (127.22.125.195:36213)
I20260928 15:59:12.743891 1963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:39635
I20260928 15:59:12.753050 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 15:59:12.777058 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.759086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48022 (local address 127.22.125.254:35703)
0928 15:59:12.759321 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:12.759329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:12.759898 (+ 569us) server_negotiation.cc:407] Connection header received
0928 15:59:12.759961 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:12.759966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:12.760102 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:12.760212 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:12.762408 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.763571 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.765869 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.766549 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.766672 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.770122 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:12.770131 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:12.770136 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:12.770174 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:12.773815 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:12.774717 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:12.774725 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:12.774729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:12.774890 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:12.775358 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:12.775363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:12.775366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:12.775723 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:12.776022 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:12.776472 (+ 450us) server_negotiation.cc:299] Negotiation successful
0928 15:59:12.776709 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 15:59:12.781320 1963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 15:59:12.788443 1963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 15:59:12.812830 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.800573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:36213 (local address 127.0.0.1:38094)
0928 15:59:12.803958 (+ 3385us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:12.803992 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:12.804333 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:12.805508 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:12.805516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:12.805550 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:59:12.806048 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.806062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.807719 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.807727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.810216 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.810231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.811711 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.811717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.811912 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.811966 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 15:59:12.812146 (+ 180us) client_negotiation.cc:240] Negotiation successful
0928 15:59:12.812370 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3132,"mutex_wait_us":3,"thread_start_us":2590,"threads_started":1}
I20260928 15:59:12.813403 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.800833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.22.125.195:36213)
0928 15:59:12.801698 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:12.801711 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:12.804200 (+ 2489us) server_negotiation.cc:407] Connection header received
0928 15:59:12.804569 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:12.804580 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:12.804917 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:12.805152 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:59:12.806281 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.807585 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.810418 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.811498 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.811706 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.811896 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:12.812293 (+ 397us) server_negotiation.cc:299] Negotiation successful
0928 15:59:12.812477 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":511,"threads_started":1}
I20260928 15:59:12.813675 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.802742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.22.125.194:34967)
0928 15:59:12.803270 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:12.803278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:12.804111 (+ 833us) server_negotiation.cc:407] Connection header received
0928 15:59:12.804433 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:12.804439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:12.804650 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:12.804866 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:59:12.805627 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.807039 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.810289 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.811842 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.811991 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.812182 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:12.813118 (+ 936us) server_negotiation.cc:299] Negotiation successful
0928 15:59:12.813318 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":453,"threads_started":1}
I20260928 15:59:12.814141 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.802440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:34967 (local address 127.0.0.1:50912)
0928 15:59:12.803952 (+ 1512us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:12.803983 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:12.804216 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:12.805044 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:12.805051 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:12.805088 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:59:12.805440 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.805449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.807154 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.807159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.810146 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.810160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.812243 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.812249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.812420 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.812450 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 15:59:12.812553 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 15:59:12.812665 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1378,"mutex_wait_us":5,"thread_start_us":4146,"threads_started":1}
I20260928 15:59:12.815546 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.801206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54620 (local address 127.22.125.193:39755)
0928 15:59:12.802307 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:12.802548 (+ 241us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:12.806933 (+ 4385us) server_negotiation.cc:407] Connection header received
0928 15:59:12.807224 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:12.807234 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:12.807460 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:12.807623 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:59:12.808748 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.810348 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.812674 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.814447 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.814645 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.814809 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:12.815031 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 15:59:12.815196 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":677,"threads_started":1}
I20260928 15:59:12.816293 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:12.803363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39755 (local address 127.0.0.1:54620)
0928 15:59:12.806819 (+ 3456us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:12.806844 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:12.807033 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:12.807947 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:12.807952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:12.807976 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:59:12.808453 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.808467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.810489 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.810494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.812486 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:12.812503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:12.814652 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:12.814658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:12.814820 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:12.814867 (+ 47us) client_negotiation.cc:769] Sending connection context
0928 15:59:12.814973 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 15:59:12.815092 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3337,"thread_start_us":2872,"threads_started":1}
I20260928 15:59:12.817900 2388 tablet_service.cc:1467] Processing CreateTablet for tablet 7e09803fc7e94e1ab759017315edf6c3 (DEFAULT_TABLE table=test-table [id=f1f92a373a604c3ea24959a05b205838]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:59:12.818066 2254 tablet_service.cc:1467] Processing CreateTablet for tablet 7e09803fc7e94e1ab759017315edf6c3 (DEFAULT_TABLE table=test-table [id=f1f92a373a604c3ea24959a05b205838]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:59:12.819191 2388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e09803fc7e94e1ab759017315edf6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:59:12.819422 2254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e09803fc7e94e1ab759017315edf6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:59:12.819684 2111 tablet_service.cc:1467] Processing CreateTablet for tablet 7e09803fc7e94e1ab759017315edf6c3 (DEFAULT_TABLE table=test-table [id=f1f92a373a604c3ea24959a05b205838]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 15:59:12.820999 2111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e09803fc7e94e1ab759017315edf6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:59:12.831346 2474 tablet_bootstrap.cc:492] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Bootstrap starting.
I20260928 15:59:12.833559 2475 tablet_bootstrap.cc:492] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Bootstrap starting.
I20260928 15:59:12.834830 2474 tablet_bootstrap.cc:654] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Neither blocks nor log segments found. Creating new log.
I20260928 15:59:12.836261 2474 log.cc:824] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Log is configured to *not* fsync() on all Append() calls
I20260928 15:59:12.836674 2476 tablet_bootstrap.cc:492] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Bootstrap starting.
I20260928 15:59:12.837095 2475 tablet_bootstrap.cc:654] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Neither blocks nor log segments found. Creating new log.
I20260928 15:59:12.838699 2475 log.cc:824] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Log is configured to *not* fsync() on all Append() calls
I20260928 15:59:12.839794 2476 tablet_bootstrap.cc:654] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Neither blocks nor log segments found. Creating new log.
I20260928 15:59:12.841034 2476 log.cc:824] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Log is configured to *not* fsync() on all Append() calls
I20260928 15:59:12.841892 2475 tablet_bootstrap.cc:492] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: No bootstrap required, opened a new log
I20260928 15:59:12.842193 2475 ts_tablet_manager.cc:1397] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 15:59:12.842252 2474 tablet_bootstrap.cc:492] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: No bootstrap required, opened a new log
I20260928 15:59:12.842572 2474 ts_tablet_manager.cc:1397] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260928 15:59:12.844132 2476 tablet_bootstrap.cc:492] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: No bootstrap required, opened a new log
I20260928 15:59:12.844800 2476 ts_tablet_manager.cc:1397] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260928 15:59:12.850028 2475 raft_consensus.cc:348] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:12.850399 2475 raft_consensus.cc:374] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:59:12.850560 2475 raft_consensus.cc:729] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d15e1bcab84948f9bcd1b1b4213c80ea, State: Initialized, Role: FOLLOWER
I20260928 15:59:12.851186 2475 consensus_queue.cc:260] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:12.852573 2319 heartbeater.cc:503] Master 127.22.125.254:35703 was elected leader, sending a full tablet report...
I20260928 15:59:12.853288 2475 ts_tablet_manager.cc:1428] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260928 15:59:12.853410 2474 raft_consensus.cc:348] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:12.853847 2474 raft_consensus.cc:374] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:59:12.853950 2474 raft_consensus.cc:729] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9125ae29485b4363bbd85ceb7a8dd631, State: Initialized, Role: FOLLOWER
I20260928 15:59:12.854530 2474 consensus_queue.cc:260] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:12.855796 2454 heartbeater.cc:503] Master 127.22.125.254:35703 was elected leader, sending a full tablet report...
I20260928 15:59:12.856369 2474 ts_tablet_manager.cc:1428] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260928 15:59:12.856526 2476 raft_consensus.cc:348] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:12.857172 2476 raft_consensus.cc:374] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:59:12.857285 2476 raft_consensus.cc:729] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f061e79d464d8e8ca242f991b36a56, State: Initialized, Role: FOLLOWER
I20260928 15:59:12.857877 2476 consensus_queue.cc:260] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:12.859622 2476 ts_tablet_manager.cc:1428] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
W20260928 15:59:12.899397 2320 tablet.cc:2369] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 15:59:12.966312 2455 tablet.cc:2369] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:59:13.028210 2481 raft_consensus.cc:482] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 15:59:13.028537 2481 raft_consensus.cc:504] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:13.030252 2481 leader_election.cc:288] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3f061e79d464d8e8ca242f991b36a56 (127.22.125.193:39755), d15e1bcab84948f9bcd1b1b4213c80ea (127.22.125.194:34967)
I20260928 15:59:13.036362 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.030436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39755 (local address 127.22.125.195:39447)
0928 15:59:13.030606 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:13.030625 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:13.030761 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:13.031216 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:13.031222 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:13.031256 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:59:13.031437 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:13.031444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.032718 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.032721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:13.034546 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:13.034560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.035776 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.035780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:13.035916 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.035939 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 15:59:13.036012 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 15:59:13.036171 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 15:59:13.036459 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.030590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:39447 (local address 127.22.125.193:39755)
0928 15:59:13.030793 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:13.030801 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:13.030825 (+ 24us) server_negotiation.cc:407] Connection header received
0928 15:59:13.030921 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:13.030925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:13.031029 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:13.031118 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:59:13.031581 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.032599 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.034690 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.035642 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.035793 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.035878 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:13.036086 (+ 208us) server_negotiation.cc:299] Negotiation successful
0928 15:59:13.036214 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260928 15:59:13.037129 2180 tablet.cc:2369] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 15:59:13.037470 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.030887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:53093 (local address 127.22.125.194:34967)
0928 15:59:13.031121 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:13.031127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:13.031606 (+ 479us) server_negotiation.cc:407] Connection header received
0928 15:59:13.031759 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:13.031764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:13.031839 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:13.031934 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 15:59:13.032445 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.033249 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.035257 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.036481 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.036634 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.036708 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:13.037085 (+ 377us) server_negotiation.cc:299] Negotiation successful
0928 15:59:13.037228 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 15:59:13.037611 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.030718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:34967 (local address 127.22.125.195:53093)
0928 15:59:13.031526 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:13.031540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:13.031640 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:13.031949 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:13.031952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:13.031968 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 15:59:13.032294 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:13.032301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.033400 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.033406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:13.035115 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:13.035124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.036751 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.036757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:13.036926 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.036956 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 15:59:13.037056 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 15:59:13.037182 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":434,"threads_started":1}
I20260928 15:59:13.037533 2131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e09803fc7e94e1ab759017315edf6c3" candidate_uuid: "9125ae29485b4363bbd85ceb7a8dd631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f061e79d464d8e8ca242f991b36a56" is_pre_election: true
I20260928 15:59:13.038136 2131 raft_consensus.cc:2454] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9125ae29485b4363bbd85ceb7a8dd631 in term 0.
I20260928 15:59:13.038820 2341 leader_election.cc:302] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9125ae29485b4363bbd85ceb7a8dd631, f3f061e79d464d8e8ca242f991b36a56; no voters:
I20260928 15:59:13.038769 2274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e09803fc7e94e1ab759017315edf6c3" candidate_uuid: "9125ae29485b4363bbd85ceb7a8dd631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" is_pre_election: true
I20260928 15:59:13.039189 2481 raft_consensus.cc:2781] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 15:59:13.039206 2274 raft_consensus.cc:2454] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9125ae29485b4363bbd85ceb7a8dd631 in term 0.
I20260928 15:59:13.039315 2481 raft_consensus.cc:482] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 15:59:13.039451 2481 raft_consensus.cc:3037] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:59:13.041952 2481 raft_consensus.cc:504] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:13.042596 2481 leader_election.cc:288] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [CANDIDATE]: Term 1 election: Requested vote from peers f3f061e79d464d8e8ca242f991b36a56 (127.22.125.193:39755), d15e1bcab84948f9bcd1b1b4213c80ea (127.22.125.194:34967)
I20260928 15:59:13.043077 2131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e09803fc7e94e1ab759017315edf6c3" candidate_uuid: "9125ae29485b4363bbd85ceb7a8dd631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f061e79d464d8e8ca242f991b36a56"
I20260928 15:59:13.043288 2131 raft_consensus.cc:3037] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:59:13.043322 2274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e09803fc7e94e1ab759017315edf6c3" candidate_uuid: "9125ae29485b4363bbd85ceb7a8dd631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea"
I20260928 15:59:13.043570 2274 raft_consensus.cc:3037] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:59:13.045591 2131 raft_consensus.cc:2454] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9125ae29485b4363bbd85ceb7a8dd631 in term 1.
I20260928 15:59:13.045897 2274 raft_consensus.cc:2454] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9125ae29485b4363bbd85ceb7a8dd631 in term 1.
I20260928 15:59:13.046000 2341 leader_election.cc:302] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9125ae29485b4363bbd85ceb7a8dd631, f3f061e79d464d8e8ca242f991b36a56; no voters:
I20260928 15:59:13.046300 2481 raft_consensus.cc:2781] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:59:13.046728 2481 raft_consensus.cc:686] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 1 LEADER]: Becoming Leader. State: Replica: 9125ae29485b4363bbd85ceb7a8dd631, State: Running, Role: LEADER
I20260928 15:59:13.047170 2481 consensus_queue.cc:237] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } }
I20260928 15:59:13.051760 1963 catalog_manager.cc:5495] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9125ae29485b4363bbd85ceb7a8dd631 (127.22.125.195). New cstate: current_term: 1 leader_uuid: "9125ae29485b4363bbd85ceb7a8dd631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9125ae29485b4363bbd85ceb7a8dd631" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 } health_report { overall_health: UNKNOWN } } }
I20260928 15:59:13.109759 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.104844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.22.125.195:36213)
0928 15:59:13.105029 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:13.105033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:13.105049 (+ 16us) server_negotiation.cc:407] Connection header received
0928 15:59:13.105134 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:13.105138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:13.105211 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:13.105339 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 15:59:13.105672 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.106389 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.107727 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.108130 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.108218 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.108664 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 15:59:13.109221 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 15:59:13.109344 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:13.109411 (+ 67us) server_negotiation.cc:299] Negotiation successful
0928 15:59:13.109472 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 15:59:13.117588 2131 raft_consensus.cc:1261] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 1 FOLLOWER]: Refusing update from remote peer 9125ae29485b4363bbd85ceb7a8dd631: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:59:13.117594 2274 raft_consensus.cc:1261] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 1 FOLLOWER]: Refusing update from remote peer 9125ae29485b4363bbd85ceb7a8dd631: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 15:59:13.118618 2484 consensus_queue.cc:1035] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d15e1bcab84948f9bcd1b1b4213c80ea" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:59:13.119000 2481 consensus_queue.cc:1035] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3f061e79d464d8e8ca242f991b36a56" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 15:59:13.135051 2487 mvcc.cc:205] Tried to move back new op lower bound from 7334343283160158208 to 7334343282885066752. Current Snapshot: MvccSnapshot[applied={T|T < 7334343283160158208}]
I20260928 15:59:13.141371 2489 mvcc.cc:205] Tried to move back new op lower bound from 7334343283160158208 to 7334343282885066752. Current Snapshot: MvccSnapshot[applied={T|T < 7334343283160158208}]
I20260928 15:59:13.155860 1963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:59:13.156144 1963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 15:59:13.159724 1963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f55f43dcda4b47eaadf0fdc3c84b65d3: Sending DeleteTablet for 3 replicas of tablet 7e09803fc7e94e1ab759017315edf6c3
I20260928 15:59:13.160794 2388 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e09803fc7e94e1ab759017315edf6c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:59:13 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:38094
I20260928 15:59:13.160898 2111 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e09803fc7e94e1ab759017315edf6c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:59:13 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:54620
I20260928 15:59:13.161204 2254 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e09803fc7e94e1ab759017315edf6c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 15:59:13 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:50912
I20260928 15:59:13.161789 2497 tablet_replica.cc:330] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: stopping tablet replica
I20260928 15:59:13.162290 2497 raft_consensus.cc:2229] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:59:13.163211 2496 tablet_replica.cc:330] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: stopping tablet replica
I20260928 15:59:13.163283 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 2043
I20260928 15:59:13.163407 2498 tablet_replica.cc:330] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: stopping tablet replica
I20260928 15:59:13.163626 2496 raft_consensus.cc:2229] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 1 LEADER]: Raft consensus shutting down.
I20260928 15:59:13.163880 2498 raft_consensus.cc:2229] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 15:59:13.164302 2496 raft_consensus.cc:2258] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:59:13.164381 2498 raft_consensus.cc:2258] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:59:13.164716 2497 raft_consensus.cc:2258] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 15:59:13.166757 2498 ts_tablet_manager.cc:1905] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:59:13.166811 2496 ts_tablet_manager.cc:1905] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:59:13.173939 2497 ts_tablet_manager.cc:1905] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 15:59:13.174106 2496 ts_tablet_manager.cc:1918] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:59:13.174335 2496 log.cc:1196] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/7e09803fc7e94e1ab759017315edf6c3
I20260928 15:59:13.174959 2496 ts_tablet_manager.cc:1939] T 7e09803fc7e94e1ab759017315edf6c3 P 9125ae29485b4363bbd85ceb7a8dd631: Deleting consensus metadata
I20260928 15:59:13.175261 2498 ts_tablet_manager.cc:1918] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:59:13.176779 1949 catalog_manager.cc:4841] TS 9125ae29485b4363bbd85ceb7a8dd631 (127.22.125.195:36213): tablet 7e09803fc7e94e1ab759017315edf6c3 (table test-table [id=f1f92a373a604c3ea24959a05b205838]) successfully deleted
I20260928 15:59:13.177454 2498 log.cc:1196] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/7e09803fc7e94e1ab759017315edf6c3
I20260928 15:59:13.178018 2498 ts_tablet_manager.cc:1939] T 7e09803fc7e94e1ab759017315edf6c3 P d15e1bcab84948f9bcd1b1b4213c80ea: Deleting consensus metadata
I20260928 15:59:13.179683 1948 catalog_manager.cc:4841] TS d15e1bcab84948f9bcd1b1b4213c80ea (127.22.125.194:34967): tablet 7e09803fc7e94e1ab759017315edf6c3 (table test-table [id=f1f92a373a604c3ea24959a05b205838]) successfully deleted
I20260928 15:59:13.182315 2497 ts_tablet_manager.cc:1918] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 15:59:13.182605 2497 log.cc:1196] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/7e09803fc7e94e1ab759017315edf6c3
I20260928 15:59:13.183128 2497 ts_tablet_manager.cc:1939] T 7e09803fc7e94e1ab759017315edf6c3 P f3f061e79d464d8e8ca242f991b36a56: Deleting consensus metadata
I20260928 15:59:13.184998 1946 catalog_manager.cc:4841] TS f3f061e79d464d8e8ca242f991b36a56 (127.22.125.193:39755): tablet 7e09803fc7e94e1ab759017315edf6c3 (table test-table [id=f1f92a373a604c3ea24959a05b205838]) successfully deleted
Sep 28 15:59:13 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 15:59:13.194133 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.165736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.22.125.193:39755)
0928 15:59:13.165917 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:13.165923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:13.175906 (+ 9983us) server_negotiation.cc:407] Connection header received
0928 15:59:13.176229 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:13.176235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:13.176317 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:13.176403 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:13.178459 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.179558 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.181643 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.182319 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.182452 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.187392 (+ 4940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:13.187414 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:13.187432 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:13.187483 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:13.191210 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:13.191879 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:13.191889 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:13.191893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:13.192144 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:13.192516 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:13.192523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:13.192527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:13.192984 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:13.193289 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:13.193494 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 15:59:13.193743 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 15:59:13.194851 2171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:59:13.286504 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 2043
I20260928 15:59:13.301147 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 2182
Sep 28 15:59:13 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 15:59:13.317927 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.301945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.22.125.194:34967)
0928 15:59:13.302116 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:13.302123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:13.302146 (+ 23us) server_negotiation.cc:407] Connection header received
0928 15:59:13.302376 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:13.302381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:13.302447 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:13.302552 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:13.303873 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.304664 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.305999 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.306403 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.306542 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.311120 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:13.311140 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:13.311161 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:13.311209 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:13.315098 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:13.315734 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:13.315744 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:13.315748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:13.315898 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:13.316316 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:13.316321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:13.316325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:13.316769 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:13.317111 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:13.317314 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 15:59:13.317532 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 15:59:13.318715 2314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:59:13.407253 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 2182
I20260928 15:59:13.424741 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 2325
Sep 28 15:59:13 dist-test-slave-9cqd krb5kdc[1482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 15:59:13.440842 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:13.425676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.22.125.195:36213)
0928 15:59:13.425867 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:13.425873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:13.425894 (+ 21us) server_negotiation.cc:407] Connection header received
0928 15:59:13.425981 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:13.425986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:13.426078 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:13.426196 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:13.427555 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.428476 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.429706 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:13.430177 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:13.430309 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:13.434753 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:13.434764 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:13.434778 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:13.434820 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:13.437904 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:13.438562 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:13.438569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:13.438571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:13.438747 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:13.439142 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:13.439147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:13.439155 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:13.439507 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:13.439837 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:13.440188 (+ 351us) server_negotiation.cc:299] Negotiation successful
0928 15:59:13.440423 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 15:59:13.441586 2449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:59:13.562861 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 2325
I20260928 15:59:13.581687 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 1930
I20260928 15:59:13.582897 1993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 15:59:13.695956 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 1930
I20260928 15:59:13.721873 23031 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 15:59:14.057935 23031 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 15:59:14.058182 23031 mini_ranger.cc:66] Stopping Ranger...
I20260928 15:59:14.404242 23031 mini_ranger.cc:68] Stopped Ranger
2026-09-28 15:59:14.404 UTC [1512] LOG: received smart shutdown request
2026-09-28 15:59:14.408 UTC [1512] LOG: background worker "logical replication launcher" (PID 1521) exited with exit code 1
2026-09-28 15:59:14.408 UTC [1516] LOG: shutting down
2026-09-28 15:59:14.560 UTC [1512] LOG: database system is shut down
2026-09-28T15:59:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49509 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 15:59:14 dist-test-slave-9cqd krb5kdc[2512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 15:59:14 dist-test-slave-9cqd krb5kdc[2512](info): set up 2 sockets
Sep 28 15:59:14 dist-test-slave-9cqd krb5kdc[2512](info): commencing operation
krb5kdc: starting...
W20260928 15:59:15.786051 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 15:59:15 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T15:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T15:59:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/rangeradmin_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/rangeradmin_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/rangerlookup_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/rangerlookup_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
I20260928 15:59:16.084952 23031 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYsP_Ep/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/postgres -l logfile start
2026-09-28 15:59:18.410 UTC [2542] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 15:59:18.410 UTC [2542] LOG: listening on IPv4 address "127.22.125.212", port 35075
2026-09-28 15:59:18.414 UTC [2542] LOG: listening on Unix socket "/tmp/.s.PGSQL.35075"
2026-09-28 15:59:18.437 UTC [2545] LOG: database system was shut down at 2026-09-28 15:59:17 UTC
2026-09-28 15:59:18.441 UTC [2542] LOG: database system is ready to accept connections
I20260928 15:59:19.449872 23031 mini_postgres.cc:96] Postgres bound to 35075
2026-09-28 15:59:19.467 UTC [2553] FATAL: database "slave" does not exist
127.22.125.212:35075 - accepting connections
I20260928 15:59:19.467640 23031 mini_ranger.cc:161] Starting Ranger...
I20260928 15:59:19.492875 23031 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 15:59:19.913825 23031 mini_ranger.cc:87] Created ranger Postgres database
I20260928 15:59:19.913950 23031 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-admin
2026-09-28 15:59:20,183 [I] DB FLAVOR :POSTGRES
2026-09-28 15:59:20,183 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 15:59:20,184 [I] Checking connection..
2026-09-28 15:59:20,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 15:59:20,416 [I] Checking connection passed.
2026-09-28 15:59:20,416 [I] --------- Verifying version history table ---------
2026-09-28 15:59:20,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 15:59:20,661 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 15:59:20,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 15:59:20,890 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 15:59:20,891 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 15:59:20,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 15:59:21.143 UTC [2617] WARNING: there is no transaction in progress
2026-09-28 15:59:21,150 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 15:59:21,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 15:59:21,408 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 15:59:21,409 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 15:59:21,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 15:59:21,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 15:59:21,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9cqd.c.gcp-upstream.internal','N') ;"
2026-09-28 15:59:22,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 15:59:22,433 [I] Table x_portal_user does not exist in database ranger
2026-09-28 15:59:22,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 15:59:22,678 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 15:59:22,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 15:59:22,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 15:59:23,186 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 15:59:23,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 15:59:24.072 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.088 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.103 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.119 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.135 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.152 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.206 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.212 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.227 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.237 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.247 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.260 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.269 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.276 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.282 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24.289 UTC [2725] WARNING: there is no transaction in progress
2026-09-28 15:59:24,899 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 15:59:24,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 15:59:25,149 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 15:59:25,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 15:59:25,409 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 15:59:25,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 15:59:25,667 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 15:59:25,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 15:59:25,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 15:59:26,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9cqd.c.gcp-upstream.internal';"
2026-09-28 15:59:26,360 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 15:59:26,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 15:59:26,607 [I] DB_PATCHES have already been applied
I20260928 15:59:26.613467 23031 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYsP_Ep/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 15:59:26.613551 23031 mini_ranger.cc:193] Using host: 127.22.125.212
I20260928 15:59:26.628083 23031 mini_ranger.cc:227] Ranger admin URL: http://127.22.125.212:38513
Sep 28, 2026 3:59:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 3:59:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 3:59:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 3:59:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 3:59:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 3:59:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 3:59:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 15:59:27.769392 23031 mini_ranger.cc:160] Time spent starting Ranger: real 8.302s user 0.008s sys 0.029s
Sep 28, 2026 3:59:47 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 3:59:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 3:59:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 3:59:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 3:59:48 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 15:59:48.604086 23031 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.125.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.125.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/rangerkms_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.125.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/rangerkms_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.125.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.125.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.125.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/HTTP_127.22.125.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 15:59:48 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611188, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 15:59:48.774660 23031 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 15:59:48.798046 23031 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 15:59:49.588511 23031 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 15:59:49.588608 23031 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms
I20260928 15:59:49.588639 23031 mini_ranger_kms.cc:227] Using postgres at 127.22.125.212:35075
2026-09-28 15:59:49,936 [I] DB FLAVOR :POSTGRES
2026-09-28 15:59:49,937 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 15:59:49,937 [I] Checking connection
2026-09-28 15:59:49,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 15:59:50,164 [I] connection success
2026-09-28 15:59:50,164 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 15:59:50,164 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 15:59:50,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 15:59:50,386 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 15:59:50,386 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 15:59:50,386 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 15:59:50,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.125.212:35075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 15:59:50,645 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 15:59:51.085616 23031 mini_ranger_kms.cc:310] Created kms service
I20260928 15:59:51.295740 23031 mini_ranger_kms.cc:326] Created kudu user
I20260928 15:59:51.342790 23031 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 15:59:51 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611188, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:51.786257 23031 mini_ranger_kms.cc:384] Added ranger policy
I20260928 15:59:51.786621 23031 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYsP_Ep/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYsP_Ep/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 15:59:51.786691 23031 mini_ranger_kms.cc:240] Using host: 127.22.125.212
I20260928 15:59:51.802261 23031 mini_ranger_kms.cc:276] Ranger KMS PID: 2908
I20260928 15:59:51.802398 23031 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.125.212:56277
Sep 28, 2026 3:59:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 3:59:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 3:59:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 3:59:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 3:59:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 3:59:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 15:59:54.090624 23031 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.316s user 0.095s sys 0.267s
I20260928 15:59:54.090873 23031 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 15:59:54.091037 23031 mini_ranger_kms.cc:401] 127.22.125.212:56277/kms/v1/keys
Sep 28 15:59:54 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790611194, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.125.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 15:59:54 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611194, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.125.212@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
Sep 28 15:59:55 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611194, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.125.212@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
Sep 28 15:59:57 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 15:59:57.307873 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42091
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:42091
--ranger_config_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 15:59:57.636701 2959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:59:57.636994 2959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:59:57.637130 2959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:59:57.644261 2959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 15:59:57.644369 2959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:59:57.644454 2959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:59:57.644541 2959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 15:59:57.644608 2959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 15:59:57.654819 2959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.125.212:56277/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:42091
--ranger_config_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:59:57.656180 2959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:59:57.657580 2959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:59:57.670953 2964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:57.671095 2965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:57.671121 2967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:59:57.671538 2959 server_base.cc:1034] running on GCE node
I20260928 15:59:57.672474 2959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:59:57.673914 2959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:59:57.675145 2959 hybrid_clock.cc:648] HybridClock initialized: now 1790611197675101 us; error 37 us; skew 500 ppm
I20260928 15:59:57.675488 2959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:59:57 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:57.683712 2959 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 15:59:57.685339 2959 webserver.cc:460] Webserver started at http://127.22.125.254:36489/ using document root <none> and password file <none>
I20260928 15:59:57.686038 2959 fs_manager.cc:359] Metadata directory not provided
I20260928 15:59:57.686203 2959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:59:57.686517 2959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 15:59:57 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611197, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:57.734687 2959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "d0f23e01e9dc49fe91664f21a9b61ddb"
format_stamp: "Formatted at 2026-09-28 15:59:57 on dist-test-slave-9cqd"
server_key: "b10af83611c3c473aa9354273164151f"
server_key_iv: "4d6c0e9b047085b5eb0c10833fe5117e"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:57.735574 2959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "d0f23e01e9dc49fe91664f21a9b61ddb"
format_stamp: "Formatted at 2026-09-28 15:59:57 on dist-test-slave-9cqd"
server_key: "b10af83611c3c473aa9354273164151f"
server_key_iv: "4d6c0e9b047085b5eb0c10833fe5117e"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:57.741185 2959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 15:59:57.757841 2974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:57.759703 2959 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260928 15:59:57.759891 2959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "d0f23e01e9dc49fe91664f21a9b61ddb"
format_stamp: "Formatted at 2026-09-28 15:59:57 on dist-test-slave-9cqd"
server_key: "b10af83611c3c473aa9354273164151f"
server_key_iv: "4d6c0e9b047085b5eb0c10833fe5117e"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:57.760155 2959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:59:57.788849 2959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:59:57.793840 2959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:59:57.794176 2959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:59:57.814354 3027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:42091 every 8 connection(s)
I20260928 15:59:57.814376 2959 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:42091
I20260928 15:59:57.816171 2959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 15:59:57.820070 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 2959
I20260928 15:59:57.820258 3028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 15:59:57.820302 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 15:59:57.821146 23031 external_mini_cluster.cc:1428] Setting key 9b20d21c3be9ee5980b97e0d1b4e3f35
I20260928 15:59:57.832919 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Bootstrap starting.
Sep 28 15:59:57 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 15:59:57.835843 3028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Neither blocks nor log segments found. Creating new log.
I20260928 15:59:57.836906 3028 log.cc:824] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Log is configured to *not* fsync() on all Append() calls
I20260928 15:59:57.840024 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: No bootstrap required, opened a new log
I20260928 15:59:57.844173 3031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:57.823256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.22.125.254:42091)
0928 15:59:57.825382 (+ 2126us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:57.825396 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:57.825437 (+ 41us) server_negotiation.cc:407] Connection header received
0928 15:59:57.827098 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:57.827119 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:57.827515 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:57.827901 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:57.829276 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:57.830577 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:57.831880 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:57.832597 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:57.832786 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:57.836575 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:57.836582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:57.836596 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:57.836634 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:57.839935 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:57.840665 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:57.840671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:57.840677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:57.840835 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:57.841241 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:57.841245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:57.841246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:57.841712 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:57.842052 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:57.842470 (+ 418us) server_negotiation.cc:299] Negotiation successful
0928 15:59:57.842835 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1920,"thread_start_us":1689,"threads_started":1}
I20260928 15:59:57.848663 3028 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 15:59:57.849023 3028 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 15:59:57.849133 3028 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f23e01e9dc49fe91664f21a9b61ddb, State: Initialized, Role: FOLLOWER
I20260928 15:59:57.849720 3028 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 15:59:57.849967 3028 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 15:59:57.850102 3028 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 15:59:57.850279 3028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 0 FOLLOWER]: Advancing to term 1
I20260928 15:59:57.852614 3028 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 15:59:57.853097 3028 leader_election.cc:302] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0f23e01e9dc49fe91664f21a9b61ddb; no voters:
I20260928 15:59:57.853683 3028 leader_election.cc:288] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 15:59:57.853809 3033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 15:59:57.854483 3033 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 LEADER]: Becoming Leader. State: Replica: d0f23e01e9dc49fe91664f21a9b61ddb, State: Running, Role: LEADER
I20260928 15:59:57.855042 3033 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 15:59:57.855774 3028 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 15:59:57.857892 3034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } } }
I20260928 15:59:57.857935 3035 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f23e01e9dc49fe91664f21a9b61ddb. Latest consensus state: current_term: 1 leader_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } } }
I20260928 15:59:57.858318 3034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: This master's current role is: LEADER
I20260928 15:59:57.858319 3035 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: This master's current role is: LEADER
I20260928 15:59:57.861151 3039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 15:59:57.866333 3039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 15:59:57.868963 3028 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 15:59:57.883015 3039 catalog_manager.cc:1347] Generated new cluster ID: 36800ef87c6b4ce1a9e74c27a5e08d2d
I20260928 15:59:57.883163 3039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 15:59:57.896245 3039 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 15:59:57.897626 3039 catalog_manager.cc:1504] Loading token signing keys...
I20260928 15:59:57.911371 3039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Generated new TSK 0
I20260928 15:59:57.912282 3039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 15:59:58.735785 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42091
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 15:59:59.108562 3073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:59:59.108863 3073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:59:59.109027 3073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:59:59.116760 3073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:59:59.116899 3073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:59:59.117085 3073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 15:59:59.128949 3073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.125.212:56277/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:59:59.130342 3073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:59:59.131644 3073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:59:59.148275 3083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:59.148569 3084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:59.153326 3086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:59:59.154256 3073 server_base.cc:1034] running on GCE node
I20260928 15:59:59.154971 3073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:59:59.156116 3073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:59:59.157420 3073 hybrid_clock.cc:648] HybridClock initialized: now 1790611199157330 us; error 121 us; skew 500 ppm
I20260928 15:59:59.157827 3073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:59.161240 3073 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 15:59:59.168336 3073 webserver.cc:460] Webserver started at http://127.22.125.193:34763/ using document root <none> and password file <none>
I20260928 15:59:59.169373 3073 fs_manager.cc:359] Metadata directory not provided
I20260928 15:59:59.169540 3073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:59:59.169849 3073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 15:59:59.187716 3073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "e0ccd824fb8d4a77900d972b2087a13e"
format_stamp: "Formatted at 2026-09-28 15:59:59 on dist-test-slave-9cqd"
server_key: "1640cfeefda7c8db8f6f25bb160c0ffe"
server_key_iv: "f03f44daaa8eb33520b9ec6313553e7d"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:59.188942 3073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "e0ccd824fb8d4a77900d972b2087a13e"
format_stamp: "Formatted at 2026-09-28 15:59:59 on dist-test-slave-9cqd"
server_key: "1640cfeefda7c8db8f6f25bb160c0ffe"
server_key_iv: "f03f44daaa8eb33520b9ec6313553e7d"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:59.209165 3073 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.004s sys 0.012s
I20260928 15:59:59.236650 3093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:59.241472 3073 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.003s sys 0.011s
I20260928 15:59:59.241760 3073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "e0ccd824fb8d4a77900d972b2087a13e"
format_stamp: "Formatted at 2026-09-28 15:59:59 on dist-test-slave-9cqd"
server_key: "1640cfeefda7c8db8f6f25bb160c0ffe"
server_key_iv: "f03f44daaa8eb33520b9ec6313553e7d"
server_key_version: "kuduclusterkey@0"
I20260928 15:59:59.242115 3073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 15:59:59.280081 3073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 15:59:59.284777 3073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 15:59:59.285085 3073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 15:59:59.287243 3073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 15:59:59.288874 3073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 15:59:59.288986 3073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:59.289120 3073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 15:59:59.289201 3073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 15:59:59.359781 3073 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:42121
I20260928 15:59:59.359763 3206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:42121 every 8 connection(s)
I20260928 15:59:59.361671 3073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 15:59:59.365276 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 3073
I20260928 15:59:59.365482 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 15:59:59.366307 23031 external_mini_cluster.cc:1428] Setting key 3c6ae5c4d78de2f1a5450f913c2625d4
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 15:59:59.414391 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:59.380419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:56427 (local address 127.22.125.254:42091)
0928 15:59:59.393036 (+ 12617us) server_negotiation.cc:206] Beginning negotiation
0928 15:59:59.393043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 15:59:59.393069 (+ 26us) server_negotiation.cc:407] Connection header received
0928 15:59:59.393142 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 15:59:59.393148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 15:59:59.393272 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 15:59:59.393405 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0928 15:59:59.396381 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:59.397615 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:59.400137 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 15:59:59.400900 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 15:59:59.401014 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:59.405802 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 15:59:59.405813 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 15:59:59.405822 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 15:59:59.405870 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 15:59:59.411034 (+ 5164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:59.411871 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:59.411877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:59.411881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:59.412017 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 15:59:59.412590 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 15:59:59.412595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 15:59:59.412599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 15:59:59.412897 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 15:59:59.413051 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 15:59:59.413773 (+ 722us) server_negotiation.cc:299] Negotiation successful
0928 15:59:59.414005 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12452,"thread_start_us":12126,"threads_started":1}
I20260928 15:59:59.415539 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 15:59:59.381086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42091 (local address 127.22.125.193:56427)
0928 15:59:59.382547 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0928 15:59:59.382598 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 15:59:59.386502 (+ 3904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 15:59:59.394441 (+ 7939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 15:59:59.394456 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 15:59:59.395182 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0928 15:59:59.396106 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:59.396138 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:59.397860 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:59.397866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:59.399933 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 15:59:59.399952 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 15:59:59.401365 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 15:59:59.401374 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 15:59:59.401735 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 15:59:59.402603 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 15:59:59.402644 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0928 15:59:59.405542 (+ 2898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 15:59:59.411199 (+ 5657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:59.411209 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:59.411230 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:59.411736 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:59.412165 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 15:59:59.412170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 15:59:59.412174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 15:59:59.412419 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 15:59:59.413037 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 15:59:59.413046 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 15:59:59.413407 (+ 361us) client_negotiation.cc:769] Sending connection context
0928 15:59:59.413676 (+ 269us) client_negotiation.cc:240] Negotiation successful
0928 15:59:59.414227 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1159,"thread_start_us":2392,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 15:59:59.417311 3207 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42091
I20260928 15:59:59.417649 3207 heartbeater.cc:464] Registering TS with master...
I20260928 15:59:59.418503 3207 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
I20260928 15:59:59.421010 2992 ts_manager.cc:194] Registered new tserver with Master: e0ccd824fb8d4a77900d972b2087a13e (127.22.125.193:42121)
I20260928 15:59:59.423051 2992 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:56427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 15:59:59.449564 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42091
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
W20260928 15:59:59.879256 3213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 15:59:59.879554 3213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 15:59:59.879717 3213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 15:59:59.888562 3213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 15:59:59.888762 3213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 15:59:59.888960 3213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 15:59:59.901945 3213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.125.212:56277/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 15:59:59.903780 3213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 15:59:59.905647 3213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 15:59:59.935127 3221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:59.937317 3220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 15:59:59.943601 3223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 15:59:59.949369 3213 server_base.cc:1034] running on GCE node
I20260928 15:59:59.950138 3213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 15:59:59.951419 3213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 15:59:59.952739 3213 hybrid_clock.cc:648] HybridClock initialized: now 1790611199952625 us; error 96 us; skew 500 ppm
I20260928 15:59:59.953121 3213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 15:59:59.957417 3213 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 15:59:59.979213 3213 webserver.cc:460] Webserver started at http://127.22.125.194:35659/ using document root <none> and password file <none>
I20260928 15:59:59.980381 3213 fs_manager.cc:359] Metadata directory not provided
I20260928 15:59:59.980595 3213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 15:59:59.981060 3213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 15:59:59 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 16:00:00.013813 3213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "958228b8e18a0c5c00829f02cc30c86b"
server_key_iv: "de95a249880febb955f0fc34c53cc944"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:00.015141 3213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "958228b8e18a0c5c00829f02cc30c86b"
server_key_iv: "de95a249880febb955f0fc34c53cc944"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:00.027874 3213 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.003s sys 0.008s
I20260928 16:00:00.084681 3230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:00.086983 3213 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.009s sys 0.010s
I20260928 16:00:00.087216 3213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "958228b8e18a0c5c00829f02cc30c86b"
server_key_iv: "de95a249880febb955f0fc34c53cc944"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:00.087523 3213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:00.133900 3213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:00.138914 3213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:00.139261 3213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:00.140437 3213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:00.142208 3213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:00.142302 3213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:00.142431 3213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:00.142493 3213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:00.178843 3213 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:40917
I20260928 16:00:00.178864 3350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:40917 every 8 connection(s)
I20260928 16:00:00.180521 3213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:00:00.186072 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 3213
I20260928 16:00:00.186301 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:00:00.186895 23031 external_mini_cluster.cc:1428] Setting key bfa80292cba026762aa8b528e61ae241
Sep 28 16:00:00 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611199, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:00.206221 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:00.183333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:42501 (local address 127.22.125.254:42091)
0928 16:00:00.183819 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:00.183824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:00.184930 (+ 1106us) server_negotiation.cc:407] Connection header received
0928 16:00:00.187480 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:00.187487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:00.187605 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:00.187731 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:00.189429 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.190496 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.192986 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.193664 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.193797 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:00.198287 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:00.198297 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:00.198305 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:00.198351 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:00.202377 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.203296 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.203304 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.203311 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.203462 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.203993 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.204001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.204007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.204417 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:00.204578 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:00.205437 (+ 859us) server_negotiation.cc:299] Negotiation successful
0928 16:00:00.205704 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":261,"threads_started":1}
I20260928 16:00:00.207178 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:00.183944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42091 (local address 127.22.125.194:42501)
0928 16:00:00.184777 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:00.184819 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:00.187131 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:00.188015 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:00.188027 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:00.188520 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:00.189244 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:00.189270 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.190693 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.190699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:00.192827 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:00.192842 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.194257 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.194263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:00.194529 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:00.195297 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:00.195325 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:00.198006 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:00.202530 (+ 4524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:00.202539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:00.202557 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:00.203150 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:00.203623 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:00.203628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:00.203631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:00.203859 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:00.204562 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:00.204570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:00.204983 (+ 413us) client_negotiation.cc:769] Sending connection context
0928 16:00:00.205277 (+ 294us) client_negotiation.cc:240] Negotiation successful
0928 16:00:00.205775 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":373,"threads_started":1}
I20260928 16:00:00.208801 3351 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42091
I20260928 16:00:00.209169 3351 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:00.210125 3351 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
I20260928 16:00:00.211831 2992 ts_manager.cc:194] Registered new tserver with Master: 892cbe1e0d434a0f96ebc4aaff3f2df7 (127.22.125.194:40917)
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
I20260928 16:00:00.212949 2992 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:42501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 16:00:00.241744 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42091
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:00.426834 3207 heartbeater.cc:503] Master 127.22.125.254:42091 was elected leader, sending a full tablet report...
W20260928 16:00:00.570281 3357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:00.570554 3357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:00.570746 3357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:00.577988 3357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:00.578105 3357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:00.578307 3357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:00:00.587929 3357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.125.212:56277/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:00.589221 3357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:00.590466 3357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:00.603451 3363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:00.603538 3362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:00.603482 3365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:00.604408 3357 server_base.cc:1034] running on GCE node
I20260928 16:00:00.604909 3357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:00.605710 3357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:00.606899 3357 hybrid_clock.cc:648] HybridClock initialized: now 1790611200606870 us; error 32 us; skew 500 ppm
I20260928 16:00:00.607152 3357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:00 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611200, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:00.610232 3357 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:00:00.611495 3357 webserver.cc:460] Webserver started at http://127.22.125.195:35809/ using document root <none> and password file <none>
I20260928 16:00:00.612221 3357 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:00.612398 3357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:00.612700 3357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 16:00:00 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611200, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 16:00:00.626233 3357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "507999b66ada42cf87fc1c5cd8807c71"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "47efc43dd563dc415019ea2c2832b18d"
server_key_iv: "7a1e7ef3cf3e582b25915d9149e20e9c"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:00.627032 3357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "507999b66ada42cf87fc1c5cd8807c71"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "47efc43dd563dc415019ea2c2832b18d"
server_key_iv: "7a1e7ef3cf3e582b25915d9149e20e9c"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:00.632282 3357 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 16:00:00.646946 3372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:00.648619 3357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:00:00.648783 3357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "507999b66ada42cf87fc1c5cd8807c71"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "47efc43dd563dc415019ea2c2832b18d"
server_key_iv: "7a1e7ef3cf3e582b25915d9149e20e9c"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:00.649017 3357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:00.671176 3357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:00.675956 3357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:00.676290 3357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:00.677443 3357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:00.679109 3357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:00.679206 3357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:00.679350 3357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:00.679414 3357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:00.714623 3357 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:38673
I20260928 16:00:00.714660 3485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:38673 every 8 connection(s)
I20260928 16:00:00.715924 3357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:00:00.723657 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 3357
I20260928 16:00:00.723848 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:00:00.724485 23031 external_mini_cluster.cc:1428] Setting key 6dc5ee17ff49f66b7a33c00602189ba7
Sep 28 16:00:00 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611200, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:00.735041 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:00.718477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:45379 (local address 127.22.125.254:42091)
0928 16:00:00.718970 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:00.718974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:00.719674 (+ 700us) server_negotiation.cc:407] Connection header received
0928 16:00:00.721453 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:00.721459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:00.721567 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:00.721688 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:00.723040 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.723847 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.726134 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.726618 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.726755 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:00.729929 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:00.729934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:00.729937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:00.729969 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:00.732455 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.733156 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.733159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.733161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.733279 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.733689 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.733692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.733694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.733949 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:00.734065 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:00.734626 (+ 561us) server_negotiation.cc:299] Negotiation successful
0928 16:00:00.734784 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":264,"threads_started":1}
I20260928 16:00:00.735939 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:00.718843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42091 (local address 127.22.125.195:45379)
0928 16:00:00.719520 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:00.719557 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:00.721191 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:00.721919 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:00.721926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:00.722318 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:00.722868 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:00.722884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.724074 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.724079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:00.725991 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:00.726007 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.726846 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.726854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:00.727082 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:00.727735 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:00.727761 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:00.729739 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:00.732609 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:00.732615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:00.732626 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:00.733021 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:00.733383 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:00.733386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:00.733388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:00.733573 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:00.734050 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:00.734055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:00.734337 (+ 282us) client_negotiation.cc:769] Sending connection context
0928 16:00:00.734543 (+ 206us) client_negotiation.cc:240] Negotiation successful
0928 16:00:00.734874 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":271,"threads_started":1}
I20260928 16:00:00.737337 3486 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42091
I20260928 16:00:00.737614 3486 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:00.738317 3486 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
I20260928 16:00:00.739827 2992 ts_manager.cc:194] Registered new tserver with Master: 507999b66ada42cf87fc1c5cd8807c71 (127.22.125.195:38673)
I20260928 16:00:00.740614 2992 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:45379
I20260928 16:00:00.751677 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:00:00.753484 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 3073
Sep 28 16:00:00 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 16:00:00.769714 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:00.754433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52338 (local address 127.22.125.193:42121)
0928 16:00:00.755066 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:00.755073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:00.755096 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:00:00.755148 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:00.755154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:00.755340 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:00.755455 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:00.756949 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.758070 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.759461 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.759976 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.760147 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:00.764392 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:00.764402 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:00.764414 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:00.764444 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:00.767439 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.768091 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.768097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.768100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.768234 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.768547 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.768550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.768552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.768860 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:00.769075 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:00.769273 (+ 198us) server_negotiation.cc:299] Negotiation successful
0928 16:00:00.769445 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":322,"threads_started":1}
I20260928 16:00:00.770956 3202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:00.853147 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 3073
I20260928 16:00:00.868994 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 3213
Sep 28 16:00:00 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 16:00:00.884702 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:00.869924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.22.125.194:40917)
0928 16:00:00.870426 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:00.870433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:00.870459 (+ 26us) server_negotiation.cc:407] Connection header received
0928 16:00:00.870518 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:00.870524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:00.870778 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:00.870956 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:00.872301 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.873262 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.874643 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.875172 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.875270 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:00.879451 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:00.879461 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:00.879474 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:00.879504 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:00.882203 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.882887 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.882893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.882896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.883023 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:00.883339 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:00.883342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:00.883344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:00.883649 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:00.883865 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:00.884207 (+ 342us) server_negotiation.cc:299] Negotiation successful
0928 16:00:00.884419 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":256,"threads_started":1}
I20260928 16:00:00.885964 3346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:00.967799 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 3213
I20260928 16:00:00.987463 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 3357
Sep 28 16:00:00 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 16:00:01.003756 3495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:00.988557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.22.125.195:38673)
0928 16:00:00.989233 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:00.989239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:00.989268 (+ 29us) server_negotiation.cc:407] Connection header received
0928 16:00:00.989325 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:00.989331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:00.989554 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:00.989676 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:00.990905 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.991921 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.993257 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:00.993806 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:00.993900 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:00.998193 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:00.998204 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:00.998217 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:00.998249 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:01.001016 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:01.001681 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:01.001688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:01.001691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:01.001811 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:01.002187 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:01.002196 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:01.002198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:01.002523 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:01.002855 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:01.003150 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 16:00:01.003414 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":366,"threads_started":1}
I20260928 16:00:01.005046 3481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:01.087571 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 3357
I20260928 16:00:01.103175 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 2959
I20260928 16:00:01.104468 3022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:01.213439 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 2959
I20260928 16:00:01.234701 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42091
--webserver_interface=127.22.125.254
--webserver_port=36489
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:42091
--ranger_config_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:01.550707 3498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:01.551045 3498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:01.551213 3498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:01.558295 3498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:01.558434 3498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:01.558542 3498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:01.558645 3498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:01.558717 3498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:01.568539 3498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:42091
--ranger_config_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=36489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:01.569805 3498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:01.571084 3498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:01.583215 3503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:01.583170 3504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:01.583261 3506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:01.584403 3498 server_base.cc:1034] running on GCE node
I20260928 16:00:01.585124 3498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:01.586470 3498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:01.587697 3498 hybrid_clock.cc:648] HybridClock initialized: now 1790611201587659 us; error 48 us; skew 500 ppm
I20260928 16:00:01.588030 3498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:01 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611201, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:01.591101 3498 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:00:01.592608 3498 webserver.cc:460] Webserver started at http://127.22.125.254:36489/ using document root <none> and password file <none>
I20260928 16:00:01.593238 3498 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:01.593397 3498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:00:01 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611201, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 16:00:01.667940 3498 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:00:01.671151 3514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:01.672665 3498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 16:00:01.672865 3498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "d0f23e01e9dc49fe91664f21a9b61ddb"
format_stamp: "Formatted at 2026-09-28 15:59:57 on dist-test-slave-9cqd"
server_key: "b10af83611c3c473aa9354273164151f"
server_key_iv: "4d6c0e9b047085b5eb0c10833fe5117e"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:01.673815 3498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:01.691951 3498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:01.696120 3498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:01.696422 3498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:01.714771 3498 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:42091
I20260928 16:00:01.714790 3565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:42091 every 8 connection(s)
I20260928 16:00:01.716354 3498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:00:01.722900 3566 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:00:01.726019 3566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Bootstrap starting.
I20260928 16:00:01.726552 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 3498
I20260928 16:00:01.727623 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:42121
--local_ip_for_outbound_sockets=127.22.125.193
--tserver_master_addrs=127.22.125.254:42091
--webserver_port=34763
--webserver_interface=127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:01.747212 3566 log.cc:824] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:01.755920 3566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:00:01.756487 3566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Bootstrap complete.
I20260928 16:00:01.765182 3566 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 16:00:01.765985 3566 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f23e01e9dc49fe91664f21a9b61ddb, State: Initialized, Role: FOLLOWER
I20260928 16:00:01.766584 3566 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 16:00:01.766803 3566 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:00:01.766914 3566 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:00:01.767099 3566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:00:01.770764 3566 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 16:00:01.771230 3566 leader_election.cc:302] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0f23e01e9dc49fe91664f21a9b61ddb; no voters:
I20260928 16:00:01.771862 3566 leader_election.cc:288] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 16:00:01.772030 3570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:00:01.773265 3570 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [term 2 LEADER]: Becoming Leader. State: Replica: d0f23e01e9dc49fe91664f21a9b61ddb, State: Running, Role: LEADER
I20260928 16:00:01.773855 3570 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } }
I20260928 16:00:01.774045 3566 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:01.784119 3572 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f23e01e9dc49fe91664f21a9b61ddb. Latest consensus state: current_term: 2 leader_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } } }
I20260928 16:00:01.784554 3572 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:01.788113 3571 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f23e01e9dc49fe91664f21a9b61ddb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42091 } } }
I20260928 16:00:01.788450 3571 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:01.788501 3581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:00:01.794088 3581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:00:01.795656 3581 catalog_manager.cc:1259] Loaded cluster ID: 36800ef87c6b4ce1a9e74c27a5e08d2d
I20260928 16:00:01.795804 3581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:00:01.799593 3581 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:00:01.801093 3566 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 16:00:01.801290 3581 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Loaded TSK: 0
I20260928 16:00:01.802970 3581 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:00:02.137923 3568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:02.138320 3568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:02.138538 3568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:02.150301 3568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:02.150488 3568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:02.150705 3568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:00:02.165973 3568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:42121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=34763
--tserver_master_addrs=127.22.125.254:42091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:02.167915 3568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:02.169766 3568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:02.197217 3606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:02.200719 3605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:02.201529 3609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:02.201901 3568 server_base.cc:1034] running on GCE node
I20260928 16:00:02.202543 3568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:02.203686 3568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:02.204949 3568 hybrid_clock.cc:648] HybridClock initialized: now 1790611202204888 us; error 53 us; skew 500 ppm
I20260928 16:00:02.205281 3568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:02 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:02.208986 3568 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:00:02.211551 3568 webserver.cc:460] Webserver started at http://127.22.125.193:34763/ using document root <none> and password file <none>
I20260928 16:00:02.212504 3568 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:02.212728 3568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:00:02 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 16:00:02.253789 3568 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.016s sys 0.003s
I20260928 16:00:02.256878 3616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:02.258405 3568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:00:02.258597 3568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "e0ccd824fb8d4a77900d972b2087a13e"
format_stamp: "Formatted at 2026-09-28 15:59:59 on dist-test-slave-9cqd"
server_key: "1640cfeefda7c8db8f6f25bb160c0ffe"
server_key_iv: "f03f44daaa8eb33520b9ec6313553e7d"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:02.259611 3568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:02.341455 3568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:02.346976 3568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:02.347389 3568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:02.349424 3568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:02.351251 3568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:02.351369 3568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:02.351573 3568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:02.351747 3568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:02.421337 3568 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:42121
I20260928 16:00:02.421391 3731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:42121 every 8 connection(s)
I20260928 16:00:02.422976 3568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:00:02.432979 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 3568
I20260928 16:00:02.434011 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:40917
--local_ip_for_outbound_sockets=127.22.125.194
--tserver_master_addrs=127.22.125.254:42091
--webserver_port=35659
--webserver_interface=127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:00:02 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:02.466019 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:02.427015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42091 (local address 127.22.125.193:54059)
0928 16:00:02.429373 (+ 2358us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:02.429422 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:02.435066 (+ 5644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:02.438370 (+ 3304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:02.438383 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:02.439065 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:02.439915 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:02.439941 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:02.442816 (+ 2875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:02.442822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:02.444314 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:02.444326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:02.445503 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:02.445510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:02.445769 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:02.446462 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:02.446488 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:02.456840 (+ 10352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:02.461336 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:02.461347 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:02.461364 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:02.461860 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:02.462311 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:02.462316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:02.462320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:02.462518 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:02.463324 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:02.463332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:02.463704 (+ 372us) client_negotiation.cc:769] Sending connection context
0928 16:00:02.464027 (+ 323us) client_negotiation.cc:240] Negotiation successful
0928 16:00:02.464631 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2072,"thread_start_us":1838,"threads_started":1}
I20260928 16:00:02.466060 3735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:02.428347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:54059 (local address 127.22.125.254:42091)
0928 16:00:02.436675 (+ 8328us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:02.436695 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:02.436746 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:00:02.437224 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:02.437246 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:02.437759 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:02.440247 (+ 2488us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:02.440705 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:02.442660 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:02.444508 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:02.445253 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:02.445463 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:02.457200 (+ 11737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:02.457210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:02.457232 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:02.457292 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:02.461178 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:02.462024 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:02.462031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:02.462036 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:02.462188 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:02.462643 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:02.462652 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:02.462654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:02.463179 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:02.463382 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:02.464545 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0928 16:00:02.464997 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8026,"thread_start_us":7771,"threads_started":1}
I20260928 16:00:02.469488 3732 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42091
I20260928 16:00:02.469806 3732 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:02.470525 3732 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
W20260928 16:00:02.473340 3732 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:42091 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 16:00:02.473591 3732 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:02.473858 3732 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
I20260928 16:00:02.474967 3732 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:02.475224 3732 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
W20260928 16:00:02.476282 3732 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 16:00:02.708455 3743 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260928 16:00:02.884444 3736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:02.884812 3736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:02.885004 3736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:02.892673 3736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:02.892794 3736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:02.892969 3736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:00:02.903646 3736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:40917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=35659
--tserver_master_addrs=127.22.125.254:42091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:02.905035 3736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:02.906334 3736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:02.924940 3750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:02.925899 3749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:02.926472 3753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:02.928202 3736 server_base.cc:1034] running on GCE node
I20260928 16:00:02.929019 3736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:02.930302 3736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:02.931669 3736 hybrid_clock.cc:648] HybridClock initialized: now 1790611202931596 us; error 55 us; skew 500 ppm
I20260928 16:00:02.932139 3736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:02 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:02.942833 3736 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:00:02.944622 3736 webserver.cc:460] Webserver started at http://127.22.125.194:35659/ using document root <none> and password file <none>
I20260928 16:00:02.945434 3736 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:02.945601 3736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:00:02 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 16:00:02.972872 3736 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 16:00:02.987462 3761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:02.989389 3736 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.007s sys 0.008s
I20260928 16:00:02.989579 3736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "958228b8e18a0c5c00829f02cc30c86b"
server_key_iv: "de95a249880febb955f0fc34c53cc944"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:02.990958 3736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:03.038408 3736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:03.043835 3736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:03.044253 3736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:03.051741 3736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:03.059887 3736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:03.060292 3736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260928 16:00:03.060475 3736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:03.060595 3736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 16:00:03 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790611203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:03.191315 3736 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:40917
I20260928 16:00:03.191345 3876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:40917 every 8 connection(s)
I20260928 16:00:03.193150 3736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:00:03.201238 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 3736
I20260928 16:00:03.201915 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.125.212:56277/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:38673
--local_ip_for_outbound_sockets=127.22.125.195
--tserver_master_addrs=127.22.125.254:42091
--webserver_port=35809
--webserver_interface=127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:00:03 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611202, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:03.223235 3879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:03.196144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:40739 (local address 127.22.125.254:42091)
0928 16:00:03.196681 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:03.196688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:03.197676 (+ 988us) server_negotiation.cc:407] Connection header received
0928 16:00:03.200198 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:03.200204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:03.200289 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:03.200427 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:03.202723 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:03.204654 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:03.207882 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:03.208663 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:03.208786 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:03.213439 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:03.213448 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:03.213456 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:03.213501 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:03.219023 (+ 5522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:03.220132 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:03.220138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:03.220142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:03.220306 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:03.221047 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:03.221053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:03.221056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:03.221432 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:03.221562 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:03.222593 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0928 16:00:03.222841 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":229,"threads_started":1}
I20260928 16:00:03.225939 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:03.196717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42091 (local address 127.22.125.194:40739)
0928 16:00:03.197487 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:03.197535 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:03.199840 (+ 2305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:03.200727 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:03.200740 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:03.201416 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:03.202266 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:03.202291 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:03.205163 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:03.205172 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:03.207698 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:03.207715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:03.209190 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:03.209199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:03.209521 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:03.210372 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:03.210410 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:03.213174 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:03.219228 (+ 6054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:03.219237 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:03.219256 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:03.219902 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:03.220478 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:03.220483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:03.220486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:03.220765 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:03.221690 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:03.221698 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:03.222148 (+ 450us) client_negotiation.cc:769] Sending connection context
0928 16:00:03.223875 (+ 1727us) client_negotiation.cc:240] Negotiation successful
0928 16:00:03.224504 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":275,"threads_started":1}
I20260928 16:00:03.227977 3877 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42091
I20260928 16:00:03.228384 3877 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:03.229290 3877 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
I20260928 16:00:03.231348 3531 ts_manager.cc:194] Registered new tserver with Master: 892cbe1e0d434a0f96ebc4aaff3f2df7 (127.22.125.194:40917)
I20260928 16:00:03.233771 3531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:40739
I20260928 16:00:03.476469 3732 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:03.476948 3732 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
I20260928 16:00:03.478498 3531 ts_manager.cc:194] Registered new tserver with Master: e0ccd824fb8d4a77900d972b2087a13e (127.22.125.193:42121)
I20260928 16:00:03.479700 3531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:54059
Sep 28 16:00:03 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
W20260928 16:00:03.647222 3881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:03.647606 3881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:03.647791 3881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:03.658671 3881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:03.658867 3881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:03.659085 3881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:00:03.673849 3881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.125.212:56277/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:38673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=35809
--tserver_master_addrs=127.22.125.254:42091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:03.675659 3881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:03.677377 3881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:03.693084 3887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:03.694741 3888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:03.696338 3890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:03.697364 3881 server_base.cc:1034] running on GCE node
I20260928 16:00:03.698159 3881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:03.699396 3881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:03.700655 3881 hybrid_clock.cc:648] HybridClock initialized: now 1790611203700589 us; error 42 us; skew 500 ppm
I20260928 16:00:03.701038 3881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:03 dist-test-slave-9cqd krb5kdc[2512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:03.705467 3881 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:00:03.707450 3881 webserver.cc:460] Webserver started at http://127.22.125.195:35809/ using document root <none> and password file <none>
I20260928 16:00:03.708386 3881 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:03.708536 3881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 16:00:03 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 16:00:03.731277 3881 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 16:00:03.737910 3898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:03.740063 3881 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260928 16:00:03.740238 3881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "507999b66ada42cf87fc1c5cd8807c71"
format_stamp: "Formatted at 2026-09-28 16:00:00 on dist-test-slave-9cqd"
server_key: "47efc43dd563dc415019ea2c2832b18d"
server_key_iv: "7a1e7ef3cf3e582b25915d9149e20e9c"
server_key_version: "kuduclusterkey@0"
I20260928 16:00:03.741626 3881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:03.767037 3881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:03.772261 3881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:03.772559 3881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:03.774809 3881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:03.776734 3881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:03.776844 3881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:03.776954 3881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:03.777099 3881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 28 16:00:03 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for HTTP/127.22.125.212@KRBTEST.COM
I20260928 16:00:04.010037 3881 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:38673
I20260928 16:00:04.010082 4012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:38673 every 8 connection(s)
I20260928 16:00:04.011438 3881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:00:04.017715 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 3881
Sep 28 16:00:04 dist-test-slave-9cqd krb5kdc[2512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611203, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:04.059018 4018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.019036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.22.125.254:42091)
0928 16:00:04.040318 (+ 21282us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.040324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.040344 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:00:04.040413 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.040419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.040511 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.040623 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:04.043180 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.044306 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.046428 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.047105 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.047228 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.051196 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:04.051202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:04.051207 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:04.051288 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:04.055018 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.055921 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.055928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.055932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.056131 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.056594 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.056598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.056601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.056982 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:04.057293 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.058550 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.058750 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21172,"spinlock_wait_cycles":9600,"thread_start_us":21017,"threads_started":1}
I20260928 16:00:04.064471 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.016350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:48415 (local address 127.22.125.254:42091)
0928 16:00:04.039006 (+ 22656us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.039015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.043717 (+ 4702us) server_negotiation.cc:407] Connection header received
0928 16:00:04.043776 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.043781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.043867 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.043962 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:04.046226 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.047462 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.050382 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.051088 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.051192 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.056590 (+ 5398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:04.056597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:04.056614 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:04.056676 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:04.061033 (+ 4357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.061992 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.061998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.062002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.062134 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.062596 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.062608 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.062612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.062860 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:04.063015 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.063855 (+ 840us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.064084 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22475,"thread_start_us":22662,"threads_started":1}
I20260928 16:00:04.065480 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.015521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42091 (local address 127.22.125.195:48415)
0928 16:00:04.039848 (+ 24327us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:04.039915 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:04.042640 (+ 2725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:04.044501 (+ 1861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:04.044514 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:04.045141 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:04.045969 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.045994 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.047741 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.047747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.050167 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.050185 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.051514 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.051521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.051838 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.053416 (+ 1578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:04.053449 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:04.056313 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:04.061194 (+ 4881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:04.061203 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:04.061221 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:04.061839 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:04.062251 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:04.062257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:04.062261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:04.062469 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:04.063000 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:04.063009 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:04.063416 (+ 407us) client_negotiation.cc:769] Sending connection context
0928 16:00:04.063687 (+ 271us) client_negotiation.cc:240] Negotiation successful
0928 16:00:04.064276 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23886,"thread_start_us":27137,"threads_started":1}
I20260928 16:00:04.067050 4013 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42091
I20260928 16:00:04.067354 4013 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:04.068111 4013 heartbeater.cc:511] Master 127.22.125.254:42091 requested a full tablet report, sending...
I20260928 16:00:04.069777 3531 ts_manager.cc:194] Registered new tserver with Master: 507999b66ada42cf87fc1c5cd8807c71 (127.22.125.195:38673)
I20260928 16:00:04.070869 3531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:48415
I20260928 16:00:04.075445 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:00:04.091730 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.078512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.22.125.254:42091)
0928 16:00:04.078695 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.078704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.079132 (+ 428us) server_negotiation.cc:407] Connection header received
0928 16:00:04.079323 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.079328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.079402 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.079502 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:04.081066 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.081770 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.083125 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.083552 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.083686 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.086305 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:04.086310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:04.086314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:04.086343 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:04.089118 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.089813 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.089819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.089823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.089952 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.090328 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.090333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.090337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.090604 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:04.090880 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.091175 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.091393 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 16:00:04.097016 3531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:00:04.107882 3531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:00:04.151284 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.123290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:42121 (local address 127.0.0.1:54870)
0928 16:00:04.132653 (+ 9363us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:04.132690 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:04.132863 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:04.139394 (+ 6531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:04.139402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:04.139432 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:04.139881 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.139896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.144018 (+ 4122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.144023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.147144 (+ 3121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.147165 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.150229 (+ 3064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.150235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.150478 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.150535 (+ 57us) client_negotiation.cc:769] Sending connection context
0928 16:00:04.150640 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 16:00:04.150826 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9086,"mutex_wait_us":1,"thread_start_us":14423,"threads_started":1}
I20260928 16:00:04.153543 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.133798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:40917 (local address 127.0.0.1:58790)
0928 16:00:04.145008 (+ 11210us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:04.145026 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:04.145149 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:04.145939 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:04.145943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:04.145968 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:04.146355 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.146367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.148350 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.148354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.150533 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.150547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.152706 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.152712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.152892 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.152922 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 16:00:04.153015 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 16:00:04.153153 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11080,"mutex_wait_us":25,"thread_start_us":10905,"threads_started":1}
I20260928 16:00:04.153899 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.123716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54870 (local address 127.22.125.193:42121)
0928 16:00:04.138804 (+ 15088us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.138813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.138844 (+ 31us) server_negotiation.cc:407] Connection header received
0928 16:00:04.138950 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.138959 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.139175 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.139397 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:04.140178 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.143834 (+ 3656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.148386 (+ 4552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.149961 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.153101 (+ 3140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.153258 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.153322 (+ 64us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.153487 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14892,"thread_start_us":15901,"threads_started":1}
I20260928 16:00:04.154307 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.134347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58790 (local address 127.22.125.194:40917)
0928 16:00:04.144232 (+ 9885us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.144242 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.145143 (+ 901us) server_negotiation.cc:407] Connection header received
0928 16:00:04.145287 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.145295 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.145530 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.145709 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:04.146556 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.148171 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.150699 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.152172 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.152305 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.152479 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.153824 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.153987 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9706,"thread_start_us":9577,"threads_started":1}
I20260928 16:00:04.157585 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.123454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:38673 (local address 127.0.0.1:34428)
0928 16:00:04.137452 (+ 13998us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:04.137476 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:04.137622 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:04.148524 (+ 10902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:04.148529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:04.148559 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:04.148983 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.148996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.151239 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.151247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.154982 (+ 3735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.154999 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.156838 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.156845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.157021 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.157051 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 16:00:04.157118 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 16:00:04.157235 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13839,"thread_start_us":8783,"threads_started":1}
I20260928 16:00:04.157882 3811 tablet_service.cc:1467] Processing CreateTablet for tablet d0af1c2f9ba2439d835dd89bab29e06a (DEFAULT_TABLE table=test-table [id=8c94373f93054193b77552404ce36aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:04.157893 3665 tablet_service.cc:1467] Processing CreateTablet for tablet d0af1c2f9ba2439d835dd89bab29e06a (DEFAULT_TABLE table=test-table [id=8c94373f93054193b77552404ce36aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:04.158696 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.123981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.22.125.195:38673)
0928 16:00:04.147869 (+ 23888us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.147877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.147907 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:00:04.147998 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.148006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.148272 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.149244 (+ 972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:04.149544 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.151080 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.155199 (+ 4119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.156601 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.157981 (+ 1380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.158136 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.158217 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.158369 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23725,"thread_start_us":23429,"threads_started":1}
I20260928 16:00:04.159339 3811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0af1c2f9ba2439d835dd89bab29e06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:04.159361 3665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0af1c2f9ba2439d835dd89bab29e06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:04.162045 3947 tablet_service.cc:1467] Processing CreateTablet for tablet d0af1c2f9ba2439d835dd89bab29e06a (DEFAULT_TABLE table=test-table [id=8c94373f93054193b77552404ce36aff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:04.163280 3947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0af1c2f9ba2439d835dd89bab29e06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:04.182379 4042 tablet_bootstrap.cc:492] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Bootstrap starting.
I20260928 16:00:04.185617 4043 tablet_bootstrap.cc:492] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Bootstrap starting.
I20260928 16:00:04.186164 4041 tablet_bootstrap.cc:492] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Bootstrap starting.
I20260928 16:00:04.189596 4043 tablet_bootstrap.cc:654] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:04.190474 4041 tablet_bootstrap.cc:654] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:04.191031 4043 log.cc:824] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:04.192006 4041 log.cc:824] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:04.200201 4042 tablet_bootstrap.cc:654] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:04.201521 4042 log.cc:824] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:04.210726 4042 tablet_bootstrap.cc:492] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: No bootstrap required, opened a new log
I20260928 16:00:04.211074 4042 ts_tablet_manager.cc:1397] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.006s
I20260928 16:00:04.219869 4041 tablet_bootstrap.cc:492] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: No bootstrap required, opened a new log
I20260928 16:00:04.220078 4043 tablet_bootstrap.cc:492] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: No bootstrap required, opened a new log
I20260928 16:00:04.220254 4041 ts_tablet_manager.cc:1397] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Time spent bootstrapping tablet: real 0.034s user 0.017s sys 0.015s
I20260928 16:00:04.220458 4043 ts_tablet_manager.cc:1397] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Time spent bootstrapping tablet: real 0.035s user 0.007s sys 0.022s
I20260928 16:00:04.221979 4042 raft_consensus.cc:348] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.222469 4042 raft_consensus.cc:374] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:04.222600 4042 raft_consensus.cc:729] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 507999b66ada42cf87fc1c5cd8807c71, State: Initialized, Role: FOLLOWER
I20260928 16:00:04.223363 4042 consensus_queue.cc:260] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.230250 4041 raft_consensus.cc:348] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.230697 4041 raft_consensus.cc:374] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:04.230822 4041 raft_consensus.cc:729] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892cbe1e0d434a0f96ebc4aaff3f2df7, State: Initialized, Role: FOLLOWER
I20260928 16:00:04.231443 4041 consensus_queue.cc:260] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.236029 4013 heartbeater.cc:503] Master 127.22.125.254:42091 was elected leader, sending a full tablet report...
I20260928 16:00:04.236794 4042 ts_tablet_manager.cc:1428] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Time spent starting tablet: real 0.026s user 0.012s sys 0.012s
I20260928 16:00:04.238309 3877 heartbeater.cc:503] Master 127.22.125.254:42091 was elected leader, sending a full tablet report...
I20260928 16:00:04.239629 4043 raft_consensus.cc:348] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.240125 4043 raft_consensus.cc:374] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:04.240255 4043 raft_consensus.cc:729] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ccd824fb8d4a77900d972b2087a13e, State: Initialized, Role: FOLLOWER
I20260928 16:00:04.240908 4043 consensus_queue.cc:260] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.242686 4041 ts_tablet_manager.cc:1428] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Time spent starting tablet: real 0.022s user 0.013s sys 0.008s
I20260928 16:00:04.243580 3732 heartbeater.cc:503] Master 127.22.125.254:42091 was elected leader, sending a full tablet report...
I20260928 16:00:04.244326 4043 ts_tablet_manager.cc:1428] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Time spent starting tablet: real 0.024s user 0.009s sys 0.008s
W20260928 16:00:04.269073 4014 tablet.cc:2369] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:04.289388 4049 raft_consensus.cc:482] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:04.289723 4049 raft_consensus.cc:504] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.291837 4049 leader_election.cc:288] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 507999b66ada42cf87fc1c5cd8807c71 (127.22.125.195:38673), 892cbe1e0d434a0f96ebc4aaff3f2df7 (127.22.125.194:40917)
I20260928 16:00:04.300405 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.292338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:59911 (local address 127.22.125.195:38673)
0928 16:00:04.292512 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.292518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.293329 (+ 811us) server_negotiation.cc:407] Connection header received
0928 16:00:04.293662 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.293669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.293750 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.293820 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:04.294625 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.295543 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.298252 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.299446 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.299605 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.299687 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.300007 (+ 320us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.300161 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:00:04.300557 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.292145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:38673 (local address 127.22.125.193:59911)
0928 16:00:04.293212 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:04.293236 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:04.293462 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:04.294024 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:04.294029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:04.294055 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:04.294475 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.294488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.295664 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.295671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.298088 (+ 2417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.298103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.299669 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.299675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.299841 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.299874 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:00:04.299969 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 16:00:04.300137 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":45,"thread_start_us":639,"threads_started":1}
I20260928 16:00:04.300894 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.292327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:42841 (local address 127.22.125.194:40917)
0928 16:00:04.292586 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.292595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.293813 (+ 1218us) server_negotiation.cc:407] Connection header received
0928 16:00:04.294014 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.294020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.294094 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.294189 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:04.294849 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.295896 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.298777 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.299702 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.299821 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.299901 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.300483 (+ 582us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.300637 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 16:00:04.301478 3968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0af1c2f9ba2439d835dd89bab29e06a" candidate_uuid: "e0ccd824fb8d4a77900d972b2087a13e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "507999b66ada42cf87fc1c5cd8807c71" is_pre_election: true
I20260928 16:00:04.301716 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.293087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:40917 (local address 127.22.125.193:42841)
0928 16:00:04.293701 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:04.293715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:04.293867 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:04.294214 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:04.294219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:04.294244 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:04.294662 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.294675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.296056 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.296062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.298593 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:04.298609 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.300114 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.300119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:04.300284 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.300317 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:00:04.301213 (+ 896us) client_negotiation.cc:240] Negotiation successful
0928 16:00:04.301343 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":480,"thread_start_us":360,"threads_started":1}
I20260928 16:00:04.302068 3968 raft_consensus.cc:2454] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0ccd824fb8d4a77900d972b2087a13e in term 0.
I20260928 16:00:04.302752 3618 leader_election.cc:302] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 507999b66ada42cf87fc1c5cd8807c71, e0ccd824fb8d4a77900d972b2087a13e; no voters:
I20260928 16:00:04.302670 3832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0af1c2f9ba2439d835dd89bab29e06a" candidate_uuid: "e0ccd824fb8d4a77900d972b2087a13e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" is_pre_election: true
I20260928 16:00:04.303149 4049 raft_consensus.cc:2781] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:00:04.303169 3832 raft_consensus.cc:2454] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0ccd824fb8d4a77900d972b2087a13e in term 0.
I20260928 16:00:04.303308 4049 raft_consensus.cc:482] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:00:04.303421 4049 raft_consensus.cc:3037] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:04.305799 4049 raft_consensus.cc:504] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.306445 4049 leader_election.cc:288] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [CANDIDATE]: Term 1 election: Requested vote from peers 507999b66ada42cf87fc1c5cd8807c71 (127.22.125.195:38673), 892cbe1e0d434a0f96ebc4aaff3f2df7 (127.22.125.194:40917)
I20260928 16:00:04.306782 3968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0af1c2f9ba2439d835dd89bab29e06a" candidate_uuid: "e0ccd824fb8d4a77900d972b2087a13e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "507999b66ada42cf87fc1c5cd8807c71"
I20260928 16:00:04.306993 3968 raft_consensus.cc:3037] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:04.307140 3832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0af1c2f9ba2439d835dd89bab29e06a" candidate_uuid: "e0ccd824fb8d4a77900d972b2087a13e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7"
I20260928 16:00:04.307348 3832 raft_consensus.cc:3037] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:04.310218 3968 raft_consensus.cc:2454] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0ccd824fb8d4a77900d972b2087a13e in term 1.
I20260928 16:00:04.310514 3832 raft_consensus.cc:2454] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0ccd824fb8d4a77900d972b2087a13e in term 1.
I20260928 16:00:04.310624 3618 leader_election.cc:302] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 507999b66ada42cf87fc1c5cd8807c71, e0ccd824fb8d4a77900d972b2087a13e; no voters:
I20260928 16:00:04.310932 4049 raft_consensus.cc:2781] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:00:04.311420 4049 raft_consensus.cc:686] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 1 LEADER]: Becoming Leader. State: Replica: e0ccd824fb8d4a77900d972b2087a13e, State: Running, Role: LEADER
I20260928 16:00:04.311949 4049 consensus_queue.cc:237] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } }
I20260928 16:00:04.316998 3529 catalog_manager.cc:5495] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e reported cstate change: term changed from 0 to 1, leader changed from <none> to e0ccd824fb8d4a77900d972b2087a13e (127.22.125.193). New cstate: current_term: 1 leader_uuid: "e0ccd824fb8d4a77900d972b2087a13e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0ccd824fb8d4a77900d972b2087a13e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 } health_report { overall_health: UNKNOWN } } }
I20260928 16:00:04.350919 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.344402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54880 (local address 127.22.125.193:42121)
0928 16:00:04.344612 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.344618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.344705 (+ 87us) server_negotiation.cc:407] Connection header received
0928 16:00:04.344901 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.344906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.345016 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.345177 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:00:04.345537 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.346715 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.348333 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.349015 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.349139 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.349631 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:00:04.350352 (+ 721us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:00:04.350501 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.350550 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.350642 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:00:04.359974 3968 raft_consensus.cc:1261] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 1 FOLLOWER]: Refusing update from remote peer e0ccd824fb8d4a77900d972b2087a13e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:04.360144 3832 raft_consensus.cc:1261] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 1 FOLLOWER]: Refusing update from remote peer e0ccd824fb8d4a77900d972b2087a13e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:04.361084 4052 consensus_queue.cc:1035] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [LEADER]: Connected to new peer: Peer: permanent_uuid: "892cbe1e0d434a0f96ebc4aaff3f2df7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 40917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:04.361500 4049 consensus_queue.cc:1035] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [LEADER]: Connected to new peer: Peer: permanent_uuid: "507999b66ada42cf87fc1c5cd8807c71" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:04.396612 3529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:04.396937 3529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:04.402297 3529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0f23e01e9dc49fe91664f21a9b61ddb: Sending DeleteTablet for 3 replicas of tablet d0af1c2f9ba2439d835dd89bab29e06a
I20260928 16:00:04.403452 3811 tablet_service.cc:1514] Processing DeleteTablet for tablet d0af1c2f9ba2439d835dd89bab29e06a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:04 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:58790
I20260928 16:00:04.403466 3665 tablet_service.cc:1514] Processing DeleteTablet for tablet d0af1c2f9ba2439d835dd89bab29e06a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:04 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:54870
I20260928 16:00:04.404611 3947 tablet_service.cc:1514] Processing DeleteTablet for tablet d0af1c2f9ba2439d835dd89bab29e06a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:04 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:34428
I20260928 16:00:04.415838 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 3568
I20260928 16:00:04.428037 4067 tablet_replica.cc:330] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: stopping tablet replica
I20260928 16:00:04.428830 4067 raft_consensus.cc:2229] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:04.429500 4065 tablet_replica.cc:330] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: stopping tablet replica
I20260928 16:00:04.429728 4066 tablet_replica.cc:330] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: stopping tablet replica
I20260928 16:00:04.429932 4065 raft_consensus.cc:2229] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:00:04.430133 4066 raft_consensus.cc:2229] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:04.430513 4067 raft_consensus.cc:2258] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:04.430655 4066 raft_consensus.cc:2258] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:04.431192 4065 raft_consensus.cc:2258] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:04.433104 4066 ts_tablet_manager.cc:1905] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:04.433463 4067 ts_tablet_manager.cc:1905] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:04.435213 4065 ts_tablet_manager.cc:1905] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:04.439939 4066 ts_tablet_manager.cc:1918] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:04.440181 4066 log.cc:1196] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/d0af1c2f9ba2439d835dd89bab29e06a
I20260928 16:00:04.440812 4066 ts_tablet_manager.cc:1939] T d0af1c2f9ba2439d835dd89bab29e06a P 892cbe1e0d434a0f96ebc4aaff3f2df7: Deleting consensus metadata
I20260928 16:00:04.442073 4067 ts_tablet_manager.cc:1918] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:04.442227 4067 log.cc:1196] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/d0af1c2f9ba2439d835dd89bab29e06a
I20260928 16:00:04.442948 4067 ts_tablet_manager.cc:1939] T d0af1c2f9ba2439d835dd89bab29e06a P 507999b66ada42cf87fc1c5cd8807c71: Deleting consensus metadata
I20260928 16:00:04.442862 3518 catalog_manager.cc:4841] TS 892cbe1e0d434a0f96ebc4aaff3f2df7 (127.22.125.194:40917): tablet d0af1c2f9ba2439d835dd89bab29e06a (table test-table [id=8c94373f93054193b77552404ce36aff]) successfully deleted
I20260928 16:00:04.444674 3516 catalog_manager.cc:4841] TS 507999b66ada42cf87fc1c5cd8807c71 (127.22.125.195:38673): tablet d0af1c2f9ba2439d835dd89bab29e06a (table test-table [id=8c94373f93054193b77552404ce36aff]) successfully deleted
I20260928 16:00:04.449811 4065 ts_tablet_manager.cc:1918] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:04.450059 4065 log.cc:1196] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/d0af1c2f9ba2439d835dd89bab29e06a
I20260928 16:00:04.450668 4065 ts_tablet_manager.cc:1939] T d0af1c2f9ba2439d835dd89bab29e06a P e0ccd824fb8d4a77900d972b2087a13e: Deleting consensus metadata
I20260928 16:00:04.452347 3517 catalog_manager.cc:4841] TS e0ccd824fb8d4a77900d972b2087a13e (127.22.125.193:42121): tablet d0af1c2f9ba2439d835dd89bab29e06a (table test-table [id=8c94373f93054193b77552404ce36aff]) successfully deleted
I20260928 16:00:04.453783 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.418045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.22.125.193:42121)
0928 16:00:04.418523 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.418529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.418548 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:00:04.418606 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.418611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.418717 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.418820 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:04.432191 (+ 13371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.433419 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.435495 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.436135 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.436247 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.439699 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:04.439711 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:04.439727 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:04.439764 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:04.444671 (+ 4907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.445570 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.445579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.445583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.445785 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.452109 (+ 6324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.452113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.452117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.452587 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:04.452833 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.453195 (+ 362us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.453444 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359}
I20260928 16:00:04.454453 3727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:04.605301 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 3568
I20260928 16:00:04.623502 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 3736
I20260928 16:00:04.638433 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.624273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.22.125.194:40917)
0928 16:00:04.624447 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.624453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.624474 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:00:04.624650 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.624655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.624757 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.624871 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:04.626126 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.627248 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.628518 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.629109 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.629222 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.631724 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:04.631741 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:04.631762 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:04.631832 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:04.635520 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.636272 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.636282 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.636285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.636447 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.636810 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.636815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.636819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.637263 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:04.637575 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.637805 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.638055 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:00:04.639204 3872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:04.734814 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 3736
I20260928 16:00:04.752506 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 3881
I20260928 16:00:04.767619 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:04.753379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34432 (local address 127.22.125.195:38673)
0928 16:00:04.753573 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:04.753578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:04.753599 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:00:04.753738 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:04.753743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:04.753828 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:04.753946 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:04.755570 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.756681 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.758070 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:04.758622 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:04.758764 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:04.761541 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:04.761558 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:04.761577 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:04.761612 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:04.764933 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.765588 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.765597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.765600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.765785 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:04.766097 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:04.766113 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:04.766116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:04.766552 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:04.766870 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:04.767047 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 16:00:04.767261 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 16:00:04.768551 4008 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:04.856822 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 3881
I20260928 16:00:04.873759 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 3498
I20260928 16:00:04.874955 3561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:04.986596 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 3498
I20260928 16:00:05.007768 23031 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 16:00:05.343504 23031 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 16:00:05.343732 23031 mini_ranger.cc:66] Stopping Ranger...
I20260928 16:00:05.689687 23031 mini_ranger.cc:68] Stopped Ranger
2026-09-28 16:00:05.689 UTC [2542] LOG: received smart shutdown request
2026-09-28 16:00:05.693 UTC [2542] LOG: background worker "logical replication launcher" (PID 2551) exited with exit code 1
2026-09-28 16:00:05.693 UTC [2546] LOG: shutting down
2026-09-28 16:00:05.841 UTC [2542] LOG: database system is shut down
2026-09-28T16:00:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51278 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:00:06 dist-test-slave-9cqd krb5kdc[4080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:00:06 dist-test-slave-9cqd krb5kdc[4080](info): set up 2 sockets
Sep 28 16:00:06 dist-test-slave-9cqd krb5kdc[4080](info): commencing operation
krb5kdc: starting...
W20260928 16:00:07.090282 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:00:07 dist-test-slave-9cqd krb5kdc[4080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:00:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:00:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:00:07.311522 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34101
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:34101,127.22.125.253:41949,127.22.125.252:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:07.633646 4094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:07.633920 4094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:07.634087 4094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:07.641234 4094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:07.641356 4094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:07.641441 4094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:07.641539 4094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:07.641620 4094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:07.651180 4094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:34101,127.22.125.253:41949,127.22.125.252:40535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:07.652473 4094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:07.653790 4094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:07.665103 4099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:07.665200 4102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:07.665200 4100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:07.666186 4094 server_base.cc:1034] running on GCE node
I20260928 16:00:07.666960 4094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:07.668298 4094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:07.669528 4094 hybrid_clock.cc:648] HybridClock initialized: now 1790611207669493 us; error 44 us; skew 500 ppm
I20260928 16:00:07.669874 4094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:07 dist-test-slave-9cqd krb5kdc[4080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:07.672782 4094 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:00:07.674203 4094 webserver.cc:460] Webserver started at http://127.22.125.254:37255/ using document root <none> and password file <none>
I20260928 16:00:07.674868 4094 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:07.675015 4094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:07.675309 4094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:07.678184 4094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "2ad7db65eba2437d9c7149cf942726d4"
format_stamp: "Formatted at 2026-09-28 16:00:07 on dist-test-slave-9cqd"
server_key: "e882239784245c21f9fbad11c403dd3a"
server_key_iv: "ed7adb197764f274dca014b7aa212a93"
server_key_version: "encryptionkey@0"
I20260928 16:00:07.678972 4094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "2ad7db65eba2437d9c7149cf942726d4"
format_stamp: "Formatted at 2026-09-28 16:00:07 on dist-test-slave-9cqd"
server_key: "e882239784245c21f9fbad11c403dd3a"
server_key_iv: "ed7adb197764f274dca014b7aa212a93"
server_key_version: "encryptionkey@0"
I20260928 16:00:07.683874 4094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260928 16:00:07.687198 4109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:07.688479 4094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:00:07.688653 4094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "2ad7db65eba2437d9c7149cf942726d4"
format_stamp: "Formatted at 2026-09-28 16:00:07 on dist-test-slave-9cqd"
server_key: "e882239784245c21f9fbad11c403dd3a"
server_key_iv: "ed7adb197764f274dca014b7aa212a93"
server_key_version: "encryptionkey@0"
I20260928 16:00:07.688856 4094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:07.705395 4094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:07.709216 4094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:07.709498 4094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:07.727869 4094 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:34101
I20260928 16:00:07.727906 4160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:34101 every 8 connection(s)
I20260928 16:00:07.729270 4094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:00:07.731194 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4094
I20260928 16:00:07.731441 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:00:07.732175 23031 external_mini_cluster.cc:1428] Setting key c2a809bdae0e760bd3d1873bee29f710
I20260928 16:00:07.733057 4161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:07.739454 4161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.253@KRBTEST.COM" created.
Sep 28 16:00:07 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:07.762116 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.742600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34101 (local address 127.0.0.1:51866)
0928 16:00:07.743692 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.743733 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:07.745503 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:07.746321 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:07.746336 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:07.747012 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:07.747345 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:07.747362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:07.748935 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:07.748941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:07.749994 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:07.750006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:07.750921 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:07.750928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:07.751099 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:07.751785 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:07.751806 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:07.754801 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:07.757946 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:07.757958 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:07.757970 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:07.758351 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:07.758749 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:07.758756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:07.758759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:07.758919 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:07.759646 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:07.759653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:07.759877 (+ 224us) client_negotiation.cc:769] Sending connection context
0928 16:00:07.760164 (+ 287us) client_negotiation.cc:240] Negotiation successful
0928 16:00:07.760786 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":183,"thread_start_us":722,"threads_started":1}
I20260928 16:00:07.762135 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.742587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.22.125.254:34101)
0928 16:00:07.743396 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:07.743414 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:07.743857 (+ 443us) server_negotiation.cc:407] Connection header received
0928 16:00:07.745859 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:07.745865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:07.746077 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:07.746304 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:07.747541 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:07.748773 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:07.750149 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:07.750680 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:07.750886 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:07.755032 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:07.755040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:07.755052 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:07.755079 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:07.757816 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:07.758493 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:07.758499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:07.758502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:07.758635 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:07.759021 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:07.759029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:07.759032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:07.759507 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:07.759718 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:07.760281 (+ 563us) server_negotiation.cc:299] Negotiation successful
0928 16:00:07.760672 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":573,"thread_start_us":371,"threads_started":1}
I20260928 16:00:07.765573 4161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:00:07.766769 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.766256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46752)
0928 16:00:07.766469 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.766604 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260928 16:00:07.767062 4110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
I20260928 16:00:07.768751 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.768271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46754)
0928 16:00:07.768447 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.768547 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 16:00:07.768978 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:41949: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab.
Entry for principal kudu/127.22.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab.
I20260928 16:00:07.780568 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:41949
--webserver_interface=127.22.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab
--principal=kudu/127.22.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:34101,127.22.125.253:41949,127.22.125.252:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:07.818360 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } attempt: 1
I20260928 16:00:07.819775 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.819158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46756)
0928 16:00:07.819435 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.819594 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260928 16:00:07.820861 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.820434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46770)
0928 16:00:07.820591 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.820676 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260928 16:00:07.821058 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:41949: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
I20260928 16:00:07.889326 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } attempt: 2
I20260928 16:00:07.890599 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.890048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46780)
0928 16:00:07.890318 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.890427 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260928 16:00:07.891669 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.891243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46790)
0928 16:00:07.891439 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.891517 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260928 16:00:07.891868 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:41949: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
I20260928 16:00:07.983139 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } attempt: 3
I20260928 16:00:07.984433 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.983873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46796)
0928 16:00:07.984118 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.984240 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260928 16:00:07.985483 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:07.985080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46804)
0928 16:00:07.985224 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:07.985320 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260928 16:00:07.985693 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:41949: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
W20260928 16:00:08.100016 4168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:08.100366 4168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:08.100455 4168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:08.107481 4168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:08.107580 4168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:08.107626 4168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:08.107676 4168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:08.107712 4168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:08.117486 4168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:34101,127.22.125.253:41949,127.22.125.252:40535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab
--principal=kudu/127.22.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:08.118768 4168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:08.119988 4168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 16:00:08.128916 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } attempt: 4
I20260928 16:00:08.130357 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.129590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46820)
0928 16:00:08.130042 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.130152 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276}
I20260928 16:00:08.131618 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.131130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46826)
0928 16:00:08.131329 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.131458 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 16:00:08.131927 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:41949: Network error: Client connection negotiation failed: client connection to 127.22.125.253:41949: connect: Connection refused (error 111)
W20260928 16:00:08.132436 4173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:08.132464 4174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:08.132918 4176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:08.133620 4168 server_base.cc:1034] running on GCE node
I20260928 16:00:08.134366 4168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:08.135608 4168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:08.136803 4168 hybrid_clock.cc:648] HybridClock initialized: now 1790611208136754 us; error 42 us; skew 500 ppm
I20260928 16:00:08.137156 4168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:08.145622 4168 init.cc:377] Logged in from keytab as kudu/127.22.125.253@KRBTEST.COM (short username kudu)
I20260928 16:00:08.147183 4168 webserver.cc:460] Webserver started at http://127.22.125.253:44125/ using document root <none> and password file <none>
I20260928 16:00:08.147886 4168 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:08.148029 4168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:08.148370 4168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:08.151329 4168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data/instance:
uuid: "73c65aa0d2ad43d5a6785144bc958cc7"
format_stamp: "Formatted at 2026-09-28 16:00:08 on dist-test-slave-9cqd"
server_key: "c58b010fb89d6520285426de9635b471"
server_key_iv: "68431a9f33d5daabfa32ff9370d3b843"
server_key_version: "encryptionkey@0"
I20260928 16:00:08.152194 4168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/wal/instance:
uuid: "73c65aa0d2ad43d5a6785144bc958cc7"
format_stamp: "Formatted at 2026-09-28 16:00:08 on dist-test-slave-9cqd"
server_key: "c58b010fb89d6520285426de9635b471"
server_key_iv: "68431a9f33d5daabfa32ff9370d3b843"
server_key_version: "encryptionkey@0"
I20260928 16:00:08.157119 4168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260928 16:00:08.160454 4183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:08.161686 4168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 16:00:08.161875 4168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/wal
uuid: "73c65aa0d2ad43d5a6785144bc958cc7"
format_stamp: "Formatted at 2026-09-28 16:00:08 on dist-test-slave-9cqd"
server_key: "c58b010fb89d6520285426de9635b471"
server_key_iv: "68431a9f33d5daabfa32ff9370d3b843"
server_key_version: "encryptionkey@0"
I20260928 16:00:08.162068 4168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:08.182821 4168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:08.186676 4168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:08.186959 4168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:08.205019 4168 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.253:41949
I20260928 16:00:08.205061 4234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.253:41949 every 8 connection(s)
I20260928 16:00:08.206334 4168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
I20260928 16:00:08.209906 4235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:08.211567 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4168
I20260928 16:00:08.211767 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-1/wal/instance
I20260928 16:00:08.212374 23031 external_mini_cluster.cc:1428] Setting key efa12b2592b74f0a027e0cf4bc1f9e5b
I20260928 16:00:08.219378 4235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.252@KRBTEST.COM" created.
I20260928 16:00:08.240670 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.222733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.22.125.254:34101)
0928 16:00:08.223020 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.223027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.224717 (+ 1690us) server_negotiation.cc:407] Connection header received
0928 16:00:08.226569 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.226576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.226662 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.226890 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.228388 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.229095 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.230704 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.231140 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.231275 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.234891 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.234900 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.234909 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.234941 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.237746 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.238456 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.238462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.238465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.238587 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.239005 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.239010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.239014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.239288 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.239449 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.240136 (+ 687us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.240355 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260928 16:00:08.241621 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.223331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34101 (local address 127.0.0.1:51870)
0928 16:00:08.224452 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.224594 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.226295 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.227048 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.227058 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.227608 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.228210 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.228228 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.229262 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.229265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.230564 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.230576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.231380 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.231387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.231605 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.232346 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.232368 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.234667 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.237901 (+ 3234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.237909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.237924 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.238319 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.238700 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.238705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.238708 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.238886 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.239419 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.239426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.239741 (+ 315us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.239996 (+ 255us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.240492 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":386,"threads_started":1}
I20260928 16:00:08.243458 4235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:00:08.258736 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.244319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46840)
0928 16:00:08.244472 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.244492 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.244590 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.245586 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.245594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.246056 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.246274 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.246288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.247589 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.247596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.249191 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.249207 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.249969 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.249980 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.250239 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.250801 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.250826 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.253194 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.256356 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.256363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.256368 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.256719 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.257083 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.257089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.257092 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.257236 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.257798 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.257805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.257948 (+ 143us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.258095 (+ 147us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.258352 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 16:00:08.258862 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.244599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.22.125.253:41949)
0928 16:00:08.245183 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.245189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.245212 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:00:08.245250 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.245255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.245422 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.245532 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.246436 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.247429 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.249321 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.249818 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.249909 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.253395 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.253406 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.253417 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.253448 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.256231 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.256849 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.256857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.256860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.256978 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.257352 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.257357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.257360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.257677 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.257895 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.258243 (+ 348us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.258488 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":319,"threads_started":1}
I20260928 16:00:08.261327 4235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:00:08.262333 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.261897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37504)
0928 16:00:08.262079 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.262167 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260928 16:00:08.262573 4184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.264240 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.263842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37508)
0928 16:00:08.263961 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.264094 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 16:00:08.264441 4235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab.
Entry for principal kudu/127.22.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab.
I20260928 16:00:08.267230 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:40535
--webserver_interface=127.22.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab
--principal=kudu/127.22.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:34101,127.22.125.253:41949,127.22.125.252:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:08.313844 4235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 1
I20260928 16:00:08.315105 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.314637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37520)
0928 16:00:08.314836 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.314956 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 16:00:08.316250 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.315850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37532)
0928 16:00:08.316026 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.316147 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:00:08.316461 4235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.384699 4235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 2
I20260928 16:00:08.385938 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.385377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37536)
0928 16:00:08.385622 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.385738 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260928 16:00:08.387014 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.386633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37546)
0928 16:00:08.386787 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.386877 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 16:00:08.387236 4235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.431300 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } attempt: 5
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:00:08.444669 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.432185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46846 (local address 127.22.125.253:41949)
0928 16:00:08.432382 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.432388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.432507 (+ 119us) server_negotiation.cc:407] Connection header received
0928 16:00:08.432727 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.432731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.432858 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.432968 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.433781 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.434482 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.435670 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.436165 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.436303 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.439355 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.439360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.439363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.439395 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.442001 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.442642 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.442649 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.442653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.442779 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.443256 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.443262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.443265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.443547 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.443706 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.444119 (+ 413us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.444340 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260928 16:00:08.444669 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.432152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46846)
0928 16:00:08.432382 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.432405 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.432582 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.433014 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.433022 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.433427 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.433603 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.433612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.434623 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.434630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.435508 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.435529 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.436315 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.436320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.436519 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.437160 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.437179 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.439164 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.442148 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.442153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.442157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.442522 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.442918 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.442923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.442926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.443139 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.443672 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.443677 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.443837 (+ 160us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.443988 (+ 151us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.444290 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 16:00:08.445885 4161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:00:08.446875 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.446511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37562)
0928 16:00:08.446669 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.446747 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 16:00:08.447762 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.447446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37566)
0928 16:00:08.447596 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.447663 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 16:00:08.447986 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.478482 4235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 3
I20260928 16:00:08.479715 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.479174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37568)
0928 16:00:08.479406 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.479518 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260928 16:00:08.480937 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.480397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37574)
0928 16:00:08.480670 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.480784 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260928 16:00:08.481160 4235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.499374 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 1
I20260928 16:00:08.500558 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.499970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37586)
0928 16:00:08.500220 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.500348 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260928 16:00:08.501681 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.501263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37594)
0928 16:00:08.501453 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.501541 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260928 16:00:08.501928 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.570133 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 2
I20260928 16:00:08.571316 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.570839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37610)
0928 16:00:08.571058 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.571147 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 16:00:08.572355 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.571993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37618)
0928 16:00:08.572168 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.572244 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260928 16:00:08.572557 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
W20260928 16:00:08.589782 4242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:08.590202 4242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:08.590392 4242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:08.597808 4242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:08.597934 4242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:08.598021 4242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:08.598121 4242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:08.598217 4242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:08.607862 4242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:34101,127.22.125.253:41949,127.22.125.252:40535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab
--principal=kudu/127.22.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:40535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:08.609169 4242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:08.610446 4242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:08.621780 4247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:08.621883 4248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:08.621908 4250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:08.622819 4242 server_base.cc:1034] running on GCE node
I20260928 16:00:08.623615 4242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:08.624375 4235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 4
I20260928 16:00:08.625187 4242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:08.625491 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.624956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37620)
0928 16:00:08.625174 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.625287 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 16:00:08.626386 4242 hybrid_clock.cc:648] HybridClock initialized: now 1790611208626334 us; error 45 us; skew 500 ppm
I20260928 16:00:08.626520 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.626184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37624)
0928 16:00:08.626316 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.626385 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260928 16:00:08.626787 4235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.626914 4242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:08.630067 4242 init.cc:377] Logged in from keytab as kudu/127.22.125.252@KRBTEST.COM (short username kudu)
I20260928 16:00:08.631482 4242 webserver.cc:460] Webserver started at http://127.22.125.252:40901/ using document root <none> and password file <none>
I20260928 16:00:08.632223 4242 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:08.632360 4242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:08.632669 4242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:08.635455 4242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data/instance:
uuid: "cf5648c7e5134fd89e6395dc6354e0b4"
format_stamp: "Formatted at 2026-09-28 16:00:08 on dist-test-slave-9cqd"
server_key: "b76f7f5c42fd16e28f039a5ee30fc240"
server_key_iv: "0a4f921f608f84c2a81bd52c330452b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:08.636307 4242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/wal/instance:
uuid: "cf5648c7e5134fd89e6395dc6354e0b4"
format_stamp: "Formatted at 2026-09-28 16:00:08 on dist-test-slave-9cqd"
server_key: "b76f7f5c42fd16e28f039a5ee30fc240"
server_key_iv: "0a4f921f608f84c2a81bd52c330452b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:08.641208 4242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 16:00:08.644439 4257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:08.645648 4242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 16:00:08.645840 4242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/wal
uuid: "cf5648c7e5134fd89e6395dc6354e0b4"
format_stamp: "Formatted at 2026-09-28 16:00:08 on dist-test-slave-9cqd"
server_key: "b76f7f5c42fd16e28f039a5ee30fc240"
server_key_iv: "0a4f921f608f84c2a81bd52c330452b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:08.646039 4242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:08.665735 4242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:08.670038 4242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:08.670332 4242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:08.678754 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 3
I20260928 16:00:08.680094 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.679463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37634)
0928 16:00:08.679753 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.679861 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260928 16:00:08.681185 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.680818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37644)
0928 16:00:08.680973 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.681059 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 16:00:08.681457 4161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:40535: Network error: Client connection negotiation failed: client connection to 127.22.125.252:40535: connect: Connection refused (error 111)
I20260928 16:00:08.688987 4242 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.252:40535
I20260928 16:00:08.689003 4308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.252:40535 every 8 connection(s)
I20260928 16:00:08.690348 4242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
I20260928 16:00:08.694190 4309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:08.696877 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4242
I20260928 16:00:08.697062 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790611005938867-23031-0/minicluster-data/master-2/wal/instance
I20260928 16:00:08.697579 23031 external_mini_cluster.cc:1428] Setting key 9d45557668d73cc8a529b074c925e86a
I20260928 16:00:08.702155 4309 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:08.718699 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.704893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.22.125.254:34101)
0928 16:00:08.705095 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.705104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.705622 (+ 518us) server_negotiation.cc:407] Connection header received
0928 16:00:08.705834 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.705840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.705916 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.706020 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.707090 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.707760 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.709194 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.709631 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.709766 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.713342 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.713349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.713352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.713382 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.715882 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.716584 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.716591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.716595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.716732 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.717144 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.717149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.717152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.717417 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.717581 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.718124 (+ 543us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.718344 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 16:00:08.719731 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.699562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.22.125.252:40535)
0928 16:00:08.700404 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.700414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.700453 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:00:08.702042 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.702062 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.702449 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.702747 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.704623 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.706340 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.707802 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.708536 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.708738 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.713007 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.713015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.713026 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.713053 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.715905 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.716596 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.716603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.716605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.716748 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.717128 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.717133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.717135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.717598 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.717926 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.718259 (+ 333us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.718637 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":646,"spinlock_wait_cycles":19968,"thread_start_us":433,"threads_started":1}
I20260928 16:00:08.719749 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.704903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34101 (local address 127.0.0.1:51874)
0928 16:00:08.705494 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.705527 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.705702 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.706077 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.706083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.706690 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.706941 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.706950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.707930 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.707936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.709050 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.709062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.709851 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.709860 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.710028 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.710647 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.710667 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.713140 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.716020 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.716030 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.716060 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.716462 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.716855 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.716859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.716863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.717022 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.717547 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.717555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.717765 (+ 210us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.718015 (+ 250us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.718488 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"spinlock_wait_cycles":13056,"thread_start_us":256,"threads_started":1}
I20260928 16:00:08.721531 4309 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:08.736568 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.722298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:41949 (local address 127.0.0.1:46858)
0928 16:00:08.722459 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.722477 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.722559 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.722887 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.722895 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.723300 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.723428 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.723436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.724813 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.724817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.726084 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.726098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.727108 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.727113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.727327 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.727922 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.727942 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.730381 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.733861 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.733866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.733869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.734369 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.734799 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.734804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.734808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.734979 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.735609 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.735616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.735783 (+ 167us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.735961 (+ 178us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.736237 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 16:00:08.736956 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.722357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.22.125.253:41949)
0928 16:00:08.722559 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.722566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.722588 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:00:08.722647 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.722652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.722726 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.722831 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.723568 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.724670 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.726243 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.726929 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.727063 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.730580 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.730588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.730595 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.730642 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.733723 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.734496 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.734503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.734508 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.734679 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.735107 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.735114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.735120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.735474 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.735626 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.736115 (+ 489us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.736336 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:00:08.738013 4309 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:00:08.740031 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.723144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.22.125.254:34101)
0928 16:00:08.723296 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.723301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.723397 (+ 96us) server_negotiation.cc:407] Connection header received
0928 16:00:08.723601 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.723607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.723668 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.723737 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.725195 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.726262 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.728192 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.728621 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.728756 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.733444 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.733454 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.733459 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.733497 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.737232 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.738063 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.738068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.738071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.738254 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.738677 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.738682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.738686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.739072 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.739346 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.739600 (+ 254us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.739765 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:00:08.751775 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.738788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37668)
0928 16:00:08.738921 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.738945 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.739025 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.739493 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.739498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.739937 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.740161 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.740177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.741269 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.741277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.742668 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.742686 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.743370 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.743379 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.743555 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.744147 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.744163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.746251 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.749242 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.749249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.749252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.749690 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.750050 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.750055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.750058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.750230 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.750894 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.750898 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.751013 (+ 115us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.751181 (+ 168us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.751491 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 16:00:08.752123 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.739034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37668 (local address 127.22.125.252:40535)
0928 16:00:08.739203 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.739209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.739225 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:00:08.739272 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.739276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.739338 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.739462 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.740314 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.741152 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.742818 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.743217 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.743310 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.746427 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.746432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.746437 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.746473 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.749126 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.749799 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.749806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.749809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.749929 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.750389 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.750394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.750397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.750758 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.750913 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.751294 (+ 381us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.751706 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":64}
I20260928 16:00:08.756385 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.741516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.22.125.253:41949)
0928 16:00:08.741662 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.741665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.741682 (+ 17us) server_negotiation.cc:407] Connection header received
0928 16:00:08.741835 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.741838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.741886 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.741958 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.743446 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.744350 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.745900 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.746321 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.746459 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.750542 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.750547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.750552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.750586 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.753740 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.754554 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.754557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.754560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.754682 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.755062 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.755065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.755067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.755297 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.755496 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.755852 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.756008 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:00:08.757683 4309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4: Bootstrap starting.
I20260928 16:00:08.760866 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:00:08.761090 4309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:08.762086 4309 log.cc:824] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:08.764991 4309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4: No bootstrap required, opened a new log
I20260928 16:00:08.776381 4309 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:08.776962 4309 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:08.777153 4309 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf5648c7e5134fd89e6395dc6354e0b4, State: Initialized, Role: FOLLOWER
I20260928 16:00:08.778795 4309 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:08.784964 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.766011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.22.125.254:34101)
0928 16:00:08.766173 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.766177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.766782 (+ 605us) server_negotiation.cc:407] Connection header received
0928 16:00:08.767076 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.767084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.767165 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.767234 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.769493 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.770591 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.772698 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.773363 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.773464 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.777304 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.777313 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.777318 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.777373 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.781506 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.782402 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.782408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.782412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.782573 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.783096 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.783101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.783104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.783478 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.783708 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.784338 (+ 630us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.784580 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 16:00:08.786355 4322 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:08.786825 4322 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:00:08.788065 4309 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:08.788370 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.766234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.22.125.253:41949)
0928 16:00:08.766424 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.766430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.767599 (+ 1169us) server_negotiation.cc:407] Connection header received
0928 16:00:08.767654 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.767660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.767732 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.767818 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.770530 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.771922 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.774681 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.775588 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.775705 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.779780 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.779788 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.779794 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.779858 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.783422 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.784271 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.784276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.784279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.784424 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.784911 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.784917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.784922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.785313 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.785565 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.787771 (+ 2206us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.787998 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:00:08.797511 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.767602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.22.125.252:40535)
0928 16:00:08.767763 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.767771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.778161 (+ 10390us) server_negotiation.cc:407] Connection header received
0928 16:00:08.778376 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.778382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.778456 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.778545 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.780610 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.782082 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.784840 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.785835 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.785942 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.789752 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.789762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.789770 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.789821 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.794166 (+ 4345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.794824 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.794829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.794852 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.795036 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.795516 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.795524 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.795529 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.795987 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.796328 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.796866 (+ 538us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.797081 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260928 16:00:08.799791 4333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:00:08.799911 4333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:00:08.795666 4242 master_runner.cc:186] Error getting master registration for 127.22.125.254:34101: OK, instance_id { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" instance_seqno: 1790611207722567 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:00:08.807261 4242 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:00:08.858775 4161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 4
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611207, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
I20260928 16:00:08.873317 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.859458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.22.125.252:40535)
0928 16:00:08.859631 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.859637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.859655 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:00:08.859808 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.859818 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.859908 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.860008 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.860988 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.862040 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.863411 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.863934 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.864112 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.867589 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.867595 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.867599 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.867635 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.870753 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.871375 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.871381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.871385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.871515 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.871892 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.871899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.871902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.872233 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.872402 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.872783 (+ 381us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.872998 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:00:08.873325 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.859394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37688)
0928 16:00:08.859578 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.859603 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.859687 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.860074 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.860080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.860451 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.860727 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.860746 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.862173 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.862179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.863258 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.863270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.864121 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.864126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.864346 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.865006 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.865024 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.867403 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.870900 (+ 3497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.870907 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.870911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.871255 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.871646 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.871651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.871656 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.871782 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.872354 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.872359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.872532 (+ 173us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.872678 (+ 146us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.872949 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260928 16:00:08.879478 4161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: Bootstrap starting.
I20260928 16:00:08.883533 4161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:08.884953 4161 log.cc:824] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:08.888008 4161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: No bootstrap required, opened a new log
I20260928 16:00:08.898902 4161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:08.899271 4161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:08.899386 4161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad7db65eba2437d9c7149cf942726d4, State: Initialized, Role: FOLLOWER
I20260928 16:00:08.900031 4161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:08.900966 4336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:08.901307 4336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:00:08.902259 4161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:00:08.911348 4348 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:00:08.911532 4348 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260928 16:00:08.922262 4263 tablet.cc:2369] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:08.910929 4094 master_runner.cc:186] Error getting master registration for 127.22.125.253:41949: OK, instance_id { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" instance_seqno: 1790611208199788 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:00:08.923328 4094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:00:08.926040 4235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } attempt: 5
Sep 28 16:00:08 dist-test-slave-9cqd krb5kdc[4080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611208, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
I20260928 16:00:08.938935 4322 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:08.939261 4322 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:08.940954 4322 leader_election.cc:288] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ad7db65eba2437d9c7149cf942726d4 (127.22.125.254:34101), 73c65aa0d2ad43d5a6785144bc958cc7 (127.22.125.253:41949)
I20260928 16:00:08.941885 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad7db65eba2437d9c7149cf942726d4" is_pre_election: true
I20260928 16:00:08.942442 4136 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf5648c7e5134fd89e6395dc6354e0b4 in term 0.
I20260928 16:00:08.943753 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.926759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:40535 (local address 127.0.0.1:37702)
0928 16:00:08.927036 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:08.927055 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:08.927180 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:08.927576 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:08.927581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:08.928207 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:08.928417 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.928429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.929935 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.929942 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.931439 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:08.931454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.932703 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.932709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:08.932943 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.933605 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:08.933626 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:08.936406 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:08.940848 (+ 4442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.940853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.940857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.941373 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.941790 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:08.941795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:08.941799 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:08.941974 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:08.942710 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:08.942715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:08.942876 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 16:00:08.943042 (+ 166us) client_negotiation.cc:240] Negotiation successful
0928 16:00:08.943322 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260928 16:00:08.944198 4259 leader_election.cc:302] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ad7db65eba2437d9c7149cf942726d4, cf5648c7e5134fd89e6395dc6354e0b4; no voters:
I20260928 16:00:08.944737 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:08.926856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37702 (local address 127.22.125.252:40535)
0928 16:00:08.927039 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:08.927046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:08.927133 (+ 87us) server_negotiation.cc:407] Connection header received
0928 16:00:08.927317 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:08.927323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:08.927410 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:08.927533 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:08.928581 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.929773 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.931611 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:08.932383 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:08.932502 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:08.936607 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:08.936615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:08.936620 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:08.936655 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:08.940676 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.941516 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.941521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.941525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.941661 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:08.942107 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:08.942112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:08.942115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:08.942476 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:08.942644 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:08.944093 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0928 16:00:08.944364 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 16:00:08.945564 4322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:00:08.945712 4322 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:00:08.945807 4322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:08.951570 4235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7: Bootstrap starting.
I20260928 16:00:08.952629 4322 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:08.955128 4322 leader_election.cc:288] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [CANDIDATE]: Term 1 election: Requested vote from peers 2ad7db65eba2437d9c7149cf942726d4 (127.22.125.254:34101), 73c65aa0d2ad43d5a6785144bc958cc7 (127.22.125.253:41949)
I20260928 16:00:08.955533 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad7db65eba2437d9c7149cf942726d4"
I20260928 16:00:08.955761 4136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:08.959151 4136 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf5648c7e5134fd89e6395dc6354e0b4 in term 1.
I20260928 16:00:08.959637 4259 leader_election.cc:302] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ad7db65eba2437d9c7149cf942726d4, cf5648c7e5134fd89e6395dc6354e0b4; no voters:
I20260928 16:00:08.959686 4235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:08.960009 4322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 1 FOLLOWER]: Leader election won for term 1
W20260928 16:00:08.962182 4115 tablet.cc:2369] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:08.963116 4235 log.cc:824] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:08.967661 4322 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [term 1 LEADER]: Becoming Leader. State: Replica: cf5648c7e5134fd89e6395dc6354e0b4, State: Running, Role: LEADER
I20260928 16:00:08.968304 4322 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:08.977996 4350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf5648c7e5134fd89e6395dc6354e0b4. Latest consensus state: current_term: 1 leader_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:08.978473 4350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:08.980433 4235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7: No bootstrap required, opened a new log
I20260928 16:00:08.997538 4353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:00:08.999737 4353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:00:09.005595 4235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:09.006017 4235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:09.006153 4235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c65aa0d2ad43d5a6785144bc958cc7, State: Initialized, Role: FOLLOWER
I20260928 16:00:09.006805 4235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } }
I20260928 16:00:09.008144 4354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:09.008843 4354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:00:09.012584 4235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:09.012643 4210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" is_pre_election: true
I20260928 16:00:09.013253 4210 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf5648c7e5134fd89e6395dc6354e0b4 in term 0.
I20260928 16:00:09.017463 4209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c65aa0d2ad43d5a6785144bc958cc7"
I20260928 16:00:09.017690 4209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:09.021047 4209 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf5648c7e5134fd89e6395dc6354e0b4 in term 1.
I20260928 16:00:09.024675 4136 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [term 1 FOLLOWER]: Refusing update from remote peer cf5648c7e5134fd89e6395dc6354e0b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:09.025638 4322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:09.031340 4209 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [term 1 FOLLOWER]: Refusing update from remote peer cf5648c7e5134fd89e6395dc6354e0b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:09.032217 4322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:09.063903 4354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf5648c7e5134fd89e6395dc6354e0b4. Latest consensus state: current_term: 1 leader_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:09.064394 4354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:00:09.067055 4355 mvcc.cc:205] Tried to move back new op lower bound from 7334343512152854528 to 7334343511939235840. Current Snapshot: MvccSnapshot[applied={T|T < 7334343512152854528}]
I20260928 16:00:09.081816 4336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf5648c7e5134fd89e6395dc6354e0b4. Latest consensus state: current_term: 1 leader_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:09.082283 4336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:00:09.082698 4350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:09.082973 4350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:09.085044 4353 catalog_manager.cc:1347] Generated new cluster ID: f6443f74f7724d7b94fd895b30649ec3
I20260928 16:00:09.085135 4353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:00:09.102859 4336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:09.103220 4336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:00:09.110219 4322 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:09.110679 4322 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4 [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:09.111759 4354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7db65eba2437d9c7149cf942726d4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34101 } } peers { permanent_uuid: "73c65aa0d2ad43d5a6785144bc958cc7" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 41949 } } peers { permanent_uuid: "cf5648c7e5134fd89e6395dc6354e0b4" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 40535 } } }
I20260928 16:00:09.112128 4354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:00:09.137413 4375 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 16:00:09.143481 4353 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:00:09.145378 4353 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:00:09.165431 4353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf5648c7e5134fd89e6395dc6354e0b4: Generated new TSK 0
I20260928 16:00:09.166566 4353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:00:09.913707 4348 catalog_manager.cc:1259] Loaded cluster ID: f6443f74f7724d7b94fd895b30649ec3
I20260928 16:00:09.913829 4348 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: loading cluster ID for follower catalog manager: success
I20260928 16:00:09.916844 4348 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: acquiring CA information for follower catalog manager: success
I20260928 16:00:09.918556 4348 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2ad7db65eba2437d9c7149cf942726d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:00:10.141746 4375 catalog_manager.cc:1259] Loaded cluster ID: f6443f74f7724d7b94fd895b30649ec3
I20260928 16:00:10.141886 4375 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7: loading cluster ID for follower catalog manager: success
I20260928 16:00:10.144661 4375 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7: acquiring CA information for follower catalog manager: success
I20260928 16:00:10.146522 4375 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73c65aa0d2ad43d5a6785144bc958cc7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:00:10.272527 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4094
I20260928 16:00:10.273869 4156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:10.405174 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4094
I20260928 16:00:10.426038 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4168
I20260928 16:00:10.427205 4230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:10.542218 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4168
I20260928 16:00:10.561547 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4242
I20260928 16:00:10.562647 4304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:10.671394 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4242
2026-09-28T16:00:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4700 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:00:10 dist-test-slave-9cqd krb5kdc[4387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:00:10 dist-test-slave-9cqd krb5kdc[4387](info): set up 2 sockets
Sep 28 16:00:10 dist-test-slave-9cqd krb5kdc[4387](info): commencing operation
krb5kdc: starting...
W20260928 16:00:11.754257 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:00:11 dist-test-slave-9cqd krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:00:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:00:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:00:11.971920 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:33829
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:33829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:12.291386 4400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:12.291723 4400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:12.291893 4400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:12.298924 4400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:12.299063 4400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:12.299175 4400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:12.299279 4400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:12.299369 4400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:12.309340 4400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:33829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:33829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:12.310638 4400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:12.311942 4400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:12.323685 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:12.323634 4406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:12.323657 4408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:12.324491 4400 server_base.cc:1034] running on GCE node
I20260928 16:00:12.325284 4400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:12.326639 4400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:12.327853 4400 hybrid_clock.cc:648] HybridClock initialized: now 1790611212327805 us; error 34 us; skew 500 ppm
I20260928 16:00:12.328316 4400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:12 dist-test-slave-9cqd krb5kdc[4387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611212, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:12.331223 4400 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:00:12.332739 4400 webserver.cc:460] Webserver started at http://127.22.125.254:39549/ using document root <none> and password file <none>
I20260928 16:00:12.333401 4400 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:12.333528 4400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:12.333832 4400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:12.336903 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "4f6fb85d83c4476ca5377cf4e88b9f37"
format_stamp: "Formatted at 2026-09-28 16:00:12 on dist-test-slave-9cqd"
server_key: "ed9ba1b976c234d7c152ed243ea89b3d"
server_key_iv: "9db1387800d501e553138e0324621e9d"
server_key_version: "encryptionkey@0"
I20260928 16:00:12.337636 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "4f6fb85d83c4476ca5377cf4e88b9f37"
format_stamp: "Formatted at 2026-09-28 16:00:12 on dist-test-slave-9cqd"
server_key: "ed9ba1b976c234d7c152ed243ea89b3d"
server_key_iv: "9db1387800d501e553138e0324621e9d"
server_key_version: "encryptionkey@0"
I20260928 16:00:12.342483 4400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260928 16:00:12.345786 4415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:12.347056 4400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:00:12.347241 4400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "4f6fb85d83c4476ca5377cf4e88b9f37"
format_stamp: "Formatted at 2026-09-28 16:00:12 on dist-test-slave-9cqd"
server_key: "ed9ba1b976c234d7c152ed243ea89b3d"
server_key_iv: "9db1387800d501e553138e0324621e9d"
server_key_version: "encryptionkey@0"
I20260928 16:00:12.347441 4400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:12.383687 4400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:12.387622 4400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:12.387907 4400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:12.406190 4400 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:33829
I20260928 16:00:12.406210 4467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:33829 every 8 connection(s)
I20260928 16:00:12.407541 4400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:00:12.411207 4468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:12.411810 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4400
I20260928 16:00:12.412026 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:00:12.412961 23031 external_mini_cluster.cc:1428] Setting key c7b18b935ce81efdeb78c70e1482b117
I20260928 16:00:12.423553 4468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37: Bootstrap starting.
Sep 28 16:00:12 dist-test-slave-9cqd krb5kdc[4387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:12.426528 4468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:12.427539 4468 log.cc:824] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:12.430271 4468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37: No bootstrap required, opened a new log
I20260928 16:00:12.434681 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:12.414850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.22.125.254:33829)
0928 16:00:12.415603 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:12.415617 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:12.415656 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:00:12.417824 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:12.417844 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:12.418198 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:12.418581 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:12.419997 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:12.421303 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:12.422703 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:12.423354 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:12.423548 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:12.427503 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:12.427510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:12.427523 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:12.427565 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:12.430704 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:12.431402 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:12.431409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:12.431415 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:12.431577 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:12.431997 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:12.432003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:12.432008 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:12.432582 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:12.432902 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:12.433358 (+ 456us) server_negotiation.cc:299] Negotiation successful
0928 16:00:12.433747 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":322,"threads_started":1}
I20260928 16:00:12.438160 4468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33829 } }
I20260928 16:00:12.438503 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:00:12.438601 4468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:12.438745 4468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f6fb85d83c4476ca5377cf4e88b9f37, State: Initialized, Role: FOLLOWER
I20260928 16:00:12.439277 4468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33829 } }
I20260928 16:00:12.439519 4468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:00:12.439664 4468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:00:12.439848 4468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:12.442252 4468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33829 } }
I20260928 16:00:12.442719 4468 leader_election.cc:302] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f6fb85d83c4476ca5377cf4e88b9f37; no voters:
I20260928 16:00:12.443303 4468 leader_election.cc:288] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:00:12.443426 4473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:00:12.444126 4473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [term 1 LEADER]: Becoming Leader. State: Replica: 4f6fb85d83c4476ca5377cf4e88b9f37, State: Running, Role: LEADER
I20260928 16:00:12.444660 4473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33829 } }
I20260928 16:00:12.445350 4468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:12.447364 4475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f6fb85d83c4476ca5377cf4e88b9f37. Latest consensus state: current_term: 1 leader_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33829 } } }
I20260928 16:00:12.447326 4474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6fb85d83c4476ca5377cf4e88b9f37" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33829 } } }
I20260928 16:00:12.447765 4474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:12.447765 4475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37 [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:12.456470 4479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:00:12.461488 4479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:00:12.469735 4479 catalog_manager.cc:1347] Generated new cluster ID: 73df0b6dcddb4ba8871bb7e725679d49
I20260928 16:00:12.469848 4479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:00:12.485237 4479 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:00:12.487013 4479 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:00:12.496732 4479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f6fb85d83c4476ca5377cf4e88b9f37: Generated new TSK 0
I20260928 16:00:12.497713 4479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:00:12.550992 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4400
I20260928 16:00:12.552263 4463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:12.646564 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4400
2026-09-28T16:00:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1970 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:00:12 dist-test-slave-9cqd krb5kdc[4496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:00:12 dist-test-slave-9cqd krb5kdc[4496](info): set up 2 sockets
Sep 28 16:00:12 dist-test-slave-9cqd krb5kdc[4496](info): commencing operation
krb5kdc: starting...
W20260928 16:00:13.750476 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:00:13 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:00:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:00:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:00:13.981717 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39405
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:39405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:14.305328 4509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:14.305605 4509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:14.305783 4509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:14.312855 4509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:14.312969 4509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:14.313059 4509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:14.313149 4509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:14.313230 4509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:14.322856 4509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:39405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:14.324187 4509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:14.325491 4509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:14.337282 4517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:14.337306 4514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:14.337337 4515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:14.338363 4509 server_base.cc:1034] running on GCE node
I20260928 16:00:14.339107 4509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:14.340603 4509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:14.341854 4509 hybrid_clock.cc:648] HybridClock initialized: now 1790611214341809 us; error 46 us; skew 500 ppm
I20260928 16:00:14.342238 4509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:14 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611214, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:14.350725 4509 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:00:14.352308 4509 webserver.cc:460] Webserver started at http://127.22.125.254:39791/ using document root <none> and password file <none>
I20260928 16:00:14.352981 4509 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:14.353133 4509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:14.353431 4509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:14.356452 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "d317f135e8f4417c9cff8e691c2f296b"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "c20080dc2e21faf65a646673e62ecc1e"
server_key_iv: "4da342ddd5325af35fdf96267f3938b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:14.357206 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "d317f135e8f4417c9cff8e691c2f296b"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "c20080dc2e21faf65a646673e62ecc1e"
server_key_iv: "4da342ddd5325af35fdf96267f3938b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:14.362191 4509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:00:14.365423 4524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:14.366734 4509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:00:14.366923 4509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "d317f135e8f4417c9cff8e691c2f296b"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "c20080dc2e21faf65a646673e62ecc1e"
server_key_iv: "4da342ddd5325af35fdf96267f3938b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:14.367110 4509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:14.395155 4509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:14.399155 4509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:14.399456 4509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:14.418241 4509 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:39405
I20260928 16:00:14.418260 4576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:39405 every 8 connection(s)
I20260928 16:00:14.419585 4509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:00:14.423189 4577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:14.425760 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4509
I20260928 16:00:14.425978 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:00:14.426824 23031 external_mini_cluster.cc:1428] Setting key e82aaaf6040bd0dc704e4c59cc04e634
I20260928 16:00:14.435684 4577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Bootstrap starting.
I20260928 16:00:14.439392 4577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:14.440784 4577 log.cc:824] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:00:14 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:14.443692 4577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: No bootstrap required, opened a new log
I20260928 16:00:14.450243 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:14.428919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34054 (local address 127.22.125.254:39405)
0928 16:00:14.429741 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:14.429752 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:14.429791 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:00:14.431391 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:14.431412 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:14.431774 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:14.432163 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:14.433814 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:14.435063 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:14.436884 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:14.437559 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:14.437690 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:14.442837 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:14.442849 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:14.442863 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:14.442902 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:14.446187 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:14.446880 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:14.446887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:14.446892 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:14.447066 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:14.447472 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:14.447476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:14.447478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:14.448016 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:14.448427 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:14.448863 (+ 436us) server_negotiation.cc:299] Negotiation successful
0928 16:00:14.449266 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":405,"threads_started":1}
I20260928 16:00:14.451529 4577 raft_consensus.cc:348] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:14.451895 4577 raft_consensus.cc:374] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:14.452020 4577 raft_consensus.cc:729] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d317f135e8f4417c9cff8e691c2f296b, State: Initialized, Role: FOLLOWER
I20260928 16:00:14.452606 4577 consensus_queue.cc:260] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:14.452869 4577 raft_consensus.cc:388] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:00:14.453037 4577 raft_consensus.cc:482] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:00:14.453222 4577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:14.455304 4577 raft_consensus.cc:504] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:14.455768 4577 leader_election.cc:302] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d317f135e8f4417c9cff8e691c2f296b; no voters:
I20260928 16:00:14.456403 4577 leader_election.cc:288] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:00:14.456534 4582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:00:14.457239 4582 raft_consensus.cc:686] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 LEADER]: Becoming Leader. State: Replica: d317f135e8f4417c9cff8e691c2f296b, State: Running, Role: LEADER
I20260928 16:00:14.457736 4582 consensus_queue.cc:237] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:14.458364 4577 sys_catalog.cc:564] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:14.460634 4583 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d317f135e8f4417c9cff8e691c2f296b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } } }
I20260928 16:00:14.460621 4584 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d317f135e8f4417c9cff8e691c2f296b. Latest consensus state: current_term: 1 leader_uuid: "d317f135e8f4417c9cff8e691c2f296b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } } }
I20260928 16:00:14.461082 4584 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:14.461082 4583 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:14.462615 4588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:00:14.468339 4588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:00:14.478235 4588 catalog_manager.cc:1347] Generated new cluster ID: 3538ffcd61bb43fc9cc9a3a108da333f
I20260928 16:00:14.478361 4588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:00:14.492415 4588 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:00:14.494259 4588 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:00:14.507052 4588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Generated new TSK 0
I20260928 16:00:14.508123 4588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 16:00:14.567974 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39405
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:14.891362 4603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:14.891623 4603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:14.891782 4603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:14.898880 4603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:14.898978 4603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:14.899120 4603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:00:14.908596 4603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:14.909853 4603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:14.911181 4603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:14.924235 4609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:14.924293 4611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:14.924710 4608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:14.924940 4603 server_base.cc:1034] running on GCE node
I20260928 16:00:14.925578 4603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:14.926548 4603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:14.927773 4603 hybrid_clock.cc:648] HybridClock initialized: now 1790611214927716 us; error 43 us; skew 500 ppm
I20260928 16:00:14.928153 4603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:14 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611214, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:14.931341 4603 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:00:14.932782 4603 webserver.cc:460] Webserver started at http://127.22.125.193:45433/ using document root <none> and password file <none>
I20260928 16:00:14.933459 4603 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:14.933605 4603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:14.933913 4603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:14.936820 4603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "38a11e64b7214e35b5dcab8d8ff9a03e"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "a0444bd797d7ae4f125070c646f452c4"
server_key_iv: "3eeaddbdcaa8020118bcbad0de98b4eb"
server_key_version: "encryptionkey@0"
I20260928 16:00:14.937597 4603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "38a11e64b7214e35b5dcab8d8ff9a03e"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "a0444bd797d7ae4f125070c646f452c4"
server_key_iv: "3eeaddbdcaa8020118bcbad0de98b4eb"
server_key_version: "encryptionkey@0"
I20260928 16:00:14.942848 4603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:00:14.946130 4619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:14.947391 4603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:00:14.947564 4603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "38a11e64b7214e35b5dcab8d8ff9a03e"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "a0444bd797d7ae4f125070c646f452c4"
server_key_iv: "3eeaddbdcaa8020118bcbad0de98b4eb"
server_key_version: "encryptionkey@0"
I20260928 16:00:14.947755 4603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:14.965154 4603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:14.969146 4603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:14.969431 4603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:14.970402 4603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:14.971860 4603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:14.971963 4603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:14.972137 4603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:14.972219 4603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:15.004890 4603 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:44197
I20260928 16:00:15.004937 4732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:44197 every 8 connection(s)
I20260928 16:00:15.006232 4603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:00:15.011833 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4603
I20260928 16:00:15.012051 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:00:15.012717 23031 external_mini_cluster.cc:1428] Setting key 8a6e61fdbdfd8465387a5aec6cde78ee
Sep 28 16:00:15 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611214, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:15.026742 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:15.008941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:60093 (local address 127.22.125.254:39405)
0928 16:00:15.009419 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:15.009423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:15.010334 (+ 911us) server_negotiation.cc:407] Connection header received
0928 16:00:15.012489 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:15.012495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:15.012588 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:15.012709 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:15.014204 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.014946 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.016587 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.016996 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.017138 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:15.020807 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:15.020812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:15.020815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:15.020853 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:15.023481 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:15.024436 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:15.024440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:15.024442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:15.024562 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:15.025053 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:15.025056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:15.025058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:15.025300 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:15.025415 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:15.026306 (+ 891us) server_negotiation.cc:299] Negotiation successful
0928 16:00:15.026482 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":271,"threads_started":1}
I20260928 16:00:15.028321 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:15.009379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:60093)
0928 16:00:15.010172 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:15.010217 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:15.012216 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:15.012961 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:15.012969 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:15.013478 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:15.014032 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:15.014047 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.015115 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.015119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:15.016438 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:15.016450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.017229 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.017237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:15.017477 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:15.018166 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:15.018192 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:15.020596 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:15.023623 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:15.023634 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:15.023653 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:15.024306 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:15.024678 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:15.024686 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:15.024692 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:15.024940 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:15.025447 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:15.025458 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:15.025906 (+ 448us) client_negotiation.cc:769] Sending connection context
0928 16:00:15.026221 (+ 315us) client_negotiation.cc:240] Negotiation successful
0928 16:00:15.026797 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":353,"threads_started":1}
I20260928 16:00:15.029901 4733 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39405
I20260928 16:00:15.030238 4733 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:15.031002 4733 heartbeater.cc:511] Master 127.22.125.254:39405 requested a full tablet report, sending...
I20260928 16:00:15.033138 4541 ts_manager.cc:194] Registered new tserver with Master: 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197)
I20260928 16:00:15.034585 4541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:60093
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 16:00:15.065114 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39405
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:15.384989 4739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:15.385299 4739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:15.385474 4739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:15.392686 4739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:15.392822 4739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:15.393013 4739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:00:15.402108 4739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:15.403334 4739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:15.404667 4739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:15.417524 4745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:15.417563 4744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:15.417593 4747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:15.429554 4739 server_base.cc:1034] running on GCE node
I20260928 16:00:15.430208 4739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:15.431205 4739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:15.432418 4739 hybrid_clock.cc:648] HybridClock initialized: now 1790611215432372 us; error 54 us; skew 500 ppm
I20260928 16:00:15.432680 4739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:15 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611215, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:15.435748 4739 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:00:15.437316 4739 webserver.cc:460] Webserver started at http://127.22.125.194:40557/ using document root <none> and password file <none>
I20260928 16:00:15.438016 4739 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:15.438150 4739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:15.438453 4739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:15.441310 4739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "f374783c869641afbd63a8d0027467b1"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "63cfac90035cd463dccac1d8495b8b98"
server_key_iv: "3d8aff717bc93b0e52e870bf68c43c74"
server_key_version: "encryptionkey@0"
I20260928 16:00:15.442127 4739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "f374783c869641afbd63a8d0027467b1"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "63cfac90035cd463dccac1d8495b8b98"
server_key_iv: "3d8aff717bc93b0e52e870bf68c43c74"
server_key_version: "encryptionkey@0"
I20260928 16:00:15.447026 4739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260928 16:00:15.450341 4754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:15.451640 4739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 16:00:15.451810 4739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "f374783c869641afbd63a8d0027467b1"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "63cfac90035cd463dccac1d8495b8b98"
server_key_iv: "3d8aff717bc93b0e52e870bf68c43c74"
server_key_version: "encryptionkey@0"
I20260928 16:00:15.452004 4739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:15.472144 4739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:15.476783 4739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:15.477082 4739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:15.478000 4739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:15.479276 4739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:15.479393 4739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:15.479547 4739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:15.479621 4739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:15.512878 4739 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:41701
I20260928 16:00:15.512966 4867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:41701 every 8 connection(s)
I20260928 16:00:15.514191 4739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:00:15.517589 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4739
I20260928 16:00:15.517796 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:00:15.518236 23031 external_mini_cluster.cc:1428] Setting key 49e586ba2976fe49f6e0ebf26371a1b2
Sep 28 16:00:15 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611215, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:15.536465 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:15.516842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:36177 (local address 127.22.125.254:39405)
0928 16:00:15.517027 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:15.517032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:15.518315 (+ 1283us) server_negotiation.cc:407] Connection header received
0928 16:00:15.520421 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:15.520426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:15.520504 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:15.520615 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:15.522100 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.523268 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.524896 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.525663 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.525803 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:15.529353 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:15.529363 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:15.529371 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:15.529416 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:15.533011 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:15.533917 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:15.533925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:15.533931 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:15.534092 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:15.534505 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:15.534513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:15.534519 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:15.534915 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:15.535072 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:15.535686 (+ 614us) server_negotiation.cc:299] Negotiation successful
0928 16:00:15.535968 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:00:15.537082 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:15.517353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:36177)
0928 16:00:15.518144 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:15.518189 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:15.520177 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:15.520841 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:15.520859 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:15.521387 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:15.521927 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:15.521944 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.523402 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.523405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:15.524738 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:15.524752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:15.525861 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:15.525866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:15.526059 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:15.526764 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:15.526789 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:15.529139 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:15.533153 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:15.533159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:15.533170 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:15.533776 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:15.534201 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:15.534204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:15.534207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:15.534380 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:15.535034 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:15.535039 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:15.535315 (+ 276us) client_negotiation.cc:769] Sending connection context
0928 16:00:15.535528 (+ 213us) client_negotiation.cc:240] Negotiation successful
0928 16:00:15.535888 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":338,"threads_started":1}
I20260928 16:00:15.538556 4868 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39405
I20260928 16:00:15.538892 4868 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:15.539590 4868 heartbeater.cc:511] Master 127.22.125.254:39405 requested a full tablet report, sending...
I20260928 16:00:15.541147 4541 ts_manager.cc:194] Registered new tserver with Master: f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701)
I20260928 16:00:15.542048 4541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:36177
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 16:00:15.571188 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39405
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:15.884547 4873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:15.884797 4873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:15.884989 4873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:15.891947 4873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:15.892120 4873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:15.892364 4873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:00:15.901628 4873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:15.902828 4873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:15.904191 4873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:15.916800 4878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:15.916898 4879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:15.916839 4881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:15.917632 4873 server_base.cc:1034] running on GCE node
I20260928 16:00:15.918239 4873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:15.919091 4873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:15.920306 4873 hybrid_clock.cc:648] HybridClock initialized: now 1790611215920272 us; error 48 us; skew 500 ppm
I20260928 16:00:15.920607 4873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:15 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611215, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:15.923508 4873 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:00:15.924918 4873 webserver.cc:460] Webserver started at http://127.22.125.195:37237/ using document root <none> and password file <none>
I20260928 16:00:15.925570 4873 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:15.925715 4873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:15.926014 4873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:15.928877 4873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "fe73d580f8ed4ea2bb4af9585b43d79e"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "116dc073588944550895b5b7b414b2a3"
server_key_iv: "72fe64f92a9eaebe4359b751ccd1d71d"
server_key_version: "encryptionkey@0"
I20260928 16:00:15.929644 4873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "fe73d580f8ed4ea2bb4af9585b43d79e"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "116dc073588944550895b5b7b414b2a3"
server_key_iv: "72fe64f92a9eaebe4359b751ccd1d71d"
server_key_version: "encryptionkey@0"
I20260928 16:00:15.934564 4873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:00:15.937764 4888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:15.939007 4873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 16:00:15.939181 4873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "fe73d580f8ed4ea2bb4af9585b43d79e"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "116dc073588944550895b5b7b414b2a3"
server_key_iv: "72fe64f92a9eaebe4359b751ccd1d71d"
server_key_version: "encryptionkey@0"
I20260928 16:00:15.939393 4873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:15.953888 4873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:15.957680 4873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:15.957959 4873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:15.958922 4873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:15.960371 4873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:15.960460 4873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:15.960558 4873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:15.960605 4873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:15.992951 4873 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:36647
I20260928 16:00:15.992995 5000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:36647 every 8 connection(s)
I20260928 16:00:15.994197 4873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:00:16.001509 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 4873
I20260928 16:00:16.001703 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:00:16.002136 23031 external_mini_cluster.cc:1428] Setting key 3b47ea5972a36e7f22bf9f9d9e3e9889
Sep 28 16:00:16 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611215, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:16.014240 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:15.996630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:60053 (local address 127.22.125.254:39405)
0928 16:00:15.996855 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:15.996861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:15.997972 (+ 1111us) server_negotiation.cc:407] Connection header received
0928 16:00:15.999682 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:15.999687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:15.999756 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:15.999845 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:16.001358 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.002189 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.004243 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.004871 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.004994 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.008564 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:16.008569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:16.008572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:16.008603 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:16.011250 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.011954 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.011964 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.011966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.012153 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.012604 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.012609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.012613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.012885 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:16.013046 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.013679 (+ 633us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.013905 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:00:16.015062 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:15.997125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:60053)
0928 16:00:15.997810 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:15.997850 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:15.999420 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:16.000085 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:16.000105 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:16.000593 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:16.001173 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.001200 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.002396 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.002401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.004035 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.004101 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.005234 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.005241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.005491 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.006167 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:16.006189 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:16.008352 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:16.011399 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:16.011407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:16.011420 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:16.011820 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:16.012263 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:16.012268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:16.012272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:16.012482 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:16.013015 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:16.013023 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:16.013331 (+ 308us) client_negotiation.cc:769] Sending connection context
0928 16:00:16.013580 (+ 249us) client_negotiation.cc:240] Negotiation successful
0928 16:00:16.014057 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":271,"threads_started":1}
I20260928 16:00:16.016402 5001 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39405
I20260928 16:00:16.016731 5001 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:16.017632 5001 heartbeater.cc:511] Master 127.22.125.254:39405 requested a full tablet report, sending...
I20260928 16:00:16.018941 4541 ts_manager.cc:194] Registered new tserver with Master: fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647)
I20260928 16:00:16.019675 4541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:60053
I20260928 16:00:16.030519 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:00:16.037570 4733 heartbeater.cc:503] Master 127.22.125.254:39405 was elected leader, sending a full tablet report...
I20260928 16:00:16.047364 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.034891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.22.125.254:39405)
0928 16:00:16.035055 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.035059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.035452 (+ 393us) server_negotiation.cc:407] Connection header received
0928 16:00:16.035573 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.035576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.035630 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.035705 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:16.037073 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.037982 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.039272 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.039697 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.039838 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.042077 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:16.042082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:16.042085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:16.042113 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:16.044728 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.045421 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.045426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.045430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.045582 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.045959 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.045966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.045974 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.046269 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:16.046525 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.046833 (+ 308us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.047042 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:00:16.051193 4541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:00:16.058039 4541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:00:16.080727 5014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.069546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:41701 (local address 127.0.0.1:35892)
0928 16:00:16.070495 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:16.070536 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:16.070708 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:16.073072 (+ 2364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:16.073083 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:16.073128 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:16.073697 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.073713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.075640 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.075646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.077873 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.077886 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.079583 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.079590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.079802 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.079858 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 16:00:16.080002 (+ 144us) client_negotiation.cc:240] Negotiation successful
0928 16:00:16.080231 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"spinlock_wait_cycles":2304,"thread_start_us":1839,"threads_started":1}
I20260928 16:00:16.081331 5013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.069669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:36647 (local address 127.0.0.1:54698)
0928 16:00:16.071684 (+ 2015us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:16.071701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:16.071811 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:16.073796 (+ 1985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:16.073800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:16.073826 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:16.074194 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.074209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.076103 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.076110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.078412 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.078430 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.080503 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.080511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.080703 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.080740 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 16:00:16.080827 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 16:00:16.080962 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1893,"thread_start_us":867,"threads_started":1}
I20260928 16:00:16.081652 5015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.069718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35892 (local address 127.22.125.194:41701)
0928 16:00:16.071343 (+ 1625us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.071355 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.071391 (+ 36us) server_negotiation.cc:407] Connection header received
0928 16:00:16.071596 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.071609 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.071929 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.073938 (+ 2009us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:16.074220 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.075490 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.078041 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.079330 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.079518 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.079698 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.080090 (+ 392us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.080278 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1416,"thread_start_us":1290,"threads_started":1}
I20260928 16:00:16.082338 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.072388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58354)
0928 16:00:16.073050 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:16.073067 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:16.073172 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:16.075111 (+ 1939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:16.075116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:16.075141 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:16.075482 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.075492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.078062 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.078067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.079650 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.079663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.081596 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.081602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.081757 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.081796 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 16:00:16.081890 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 16:00:16.082012 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":331,"threads_started":1}
I20260928 16:00:16.082396 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.070264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.22.125.195:36647)
0928 16:00:16.072421 (+ 2157us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.072429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.072459 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:00:16.072566 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.072576 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.073583 (+ 1007us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.073815 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:16.074430 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.075924 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.078584 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.079933 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.080168 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.080338 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.081893 (+ 1555us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.082052 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1989,"thread_start_us":1829,"threads_started":1}
I20260928 16:00:16.082523 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.072774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.22.125.193:44197)
0928 16:00:16.074509 (+ 1735us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.074521 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.074549 (+ 28us) server_negotiation.cc:407] Connection header received
0928 16:00:16.074640 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.074667 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.074910 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.075095 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:16.076176 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.077886 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.079849 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.081382 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.081604 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.081763 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.082038 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.082204 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1566,"thread_start_us":1408,"threads_started":1}
I20260928 16:00:16.085680 4802 tablet_service.cc:1467] Processing CreateTablet for tablet 4280629655dd4a24847f899af71592a4 (DEFAULT_TABLE table=test-table [id=02c3c2e110e346a3a795875097294ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:16.085681 4936 tablet_service.cc:1467] Processing CreateTablet for tablet 4280629655dd4a24847f899af71592a4 (DEFAULT_TABLE table=test-table [id=02c3c2e110e346a3a795875097294ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:16.086279 4667 tablet_service.cc:1467] Processing CreateTablet for tablet 4280629655dd4a24847f899af71592a4 (DEFAULT_TABLE table=test-table [id=02c3c2e110e346a3a795875097294ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:16.087093 4936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4280629655dd4a24847f899af71592a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:16.087116 4802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4280629655dd4a24847f899af71592a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:16.087524 4667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4280629655dd4a24847f899af71592a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:16.097776 5019 tablet_bootstrap.cc:492] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Bootstrap starting.
I20260928 16:00:16.101236 5020 tablet_bootstrap.cc:492] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Bootstrap starting.
I20260928 16:00:16.101548 5019 tablet_bootstrap.cc:654] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:16.102928 5019 log.cc:824] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:16.104851 5020 tablet_bootstrap.cc:654] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:16.104359 5021 tablet_bootstrap.cc:492] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Bootstrap starting.
I20260928 16:00:16.106055 5020 log.cc:824] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:16.106230 5019 tablet_bootstrap.cc:492] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: No bootstrap required, opened a new log
I20260928 16:00:16.106505 5019 ts_tablet_manager.cc:1397] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 16:00:16.107800 5021 tablet_bootstrap.cc:654] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:16.109043 5020 tablet_bootstrap.cc:492] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: No bootstrap required, opened a new log
I20260928 16:00:16.109043 5021 log.cc:824] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:16.109294 5020 ts_tablet_manager.cc:1397] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 16:00:16.111892 5021 tablet_bootstrap.cc:492] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: No bootstrap required, opened a new log
I20260928 16:00:16.112151 5021 ts_tablet_manager.cc:1397] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260928 16:00:16.114696 5019 raft_consensus.cc:348] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.115056 5019 raft_consensus.cc:374] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:16.115159 5019 raft_consensus.cc:729] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe73d580f8ed4ea2bb4af9585b43d79e, State: Initialized, Role: FOLLOWER
I20260928 16:00:16.115747 5019 consensus_queue.cc:260] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.116807 5001 heartbeater.cc:503] Master 127.22.125.254:39405 was elected leader, sending a full tablet report...
I20260928 16:00:16.117400 5019 ts_tablet_manager.cc:1428] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260928 16:00:16.119742 5020 raft_consensus.cc:348] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.120103 5020 raft_consensus.cc:374] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:16.120224 5020 raft_consensus.cc:729] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f374783c869641afbd63a8d0027467b1, State: Initialized, Role: FOLLOWER
I20260928 16:00:16.120906 5020 consensus_queue.cc:260] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.121959 4868 heartbeater.cc:503] Master 127.22.125.254:39405 was elected leader, sending a full tablet report...
I20260928 16:00:16.122162 5021 raft_consensus.cc:348] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.122535 5021 raft_consensus.cc:374] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:16.122634 5021 raft_consensus.cc:729] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a11e64b7214e35b5dcab8d8ff9a03e, State: Initialized, Role: FOLLOWER
I20260928 16:00:16.122659 5020 ts_tablet_manager.cc:1428] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260928 16:00:16.123247 5021 consensus_queue.cc:260] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.124814 5021 ts_tablet_manager.cc:1428] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260928 16:00:16.232750 5025 raft_consensus.cc:482] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:16.233059 5025 raft_consensus.cc:504] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.234760 5025 leader_election.cc:288] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701), 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197)
I20260928 16:00:16.241510 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.235047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:41701 (local address 127.22.125.195:39241)
0928 16:00:16.235316 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:16.235340 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:16.235474 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:16.235977 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:16.235985 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:16.236020 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:16.236254 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.236267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.237529 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.237532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.239537 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.239553 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.240916 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.240920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.241047 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.241071 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 16:00:16.241143 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 16:00:16.241227 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260928 16:00:16.241562 5015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.235214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:39241 (local address 127.22.125.194:41701)
0928 16:00:16.235460 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.235466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.235486 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:00:16.235731 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.235738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.235833 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.235930 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:16.236401 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.237391 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.239679 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.240772 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.240936 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.241010 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.241195 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.241322 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 16:00:16.242225 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.235233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.22.125.195:37697)
0928 16:00:16.236004 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:16.236021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:16.236161 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:16.236501 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:16.236504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:16.236524 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:16.236788 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.236794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.237987 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.237996 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.239908 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:16.239917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.241457 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.241462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:16.241638 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.241676 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:00:16.241797 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 16:00:16.241916 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":418,"threads_started":1}
I20260928 16:00:16.242225 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.235537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:37697 (local address 127.22.125.193:44197)
0928 16:00:16.235768 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.235776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.236137 (+ 361us) server_negotiation.cc:407] Connection header received
0928 16:00:16.236295 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.236298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.236389 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.236485 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:16.236951 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.237837 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.240076 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.241251 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.241404 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.241474 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.241825 (+ 351us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.241999 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:00:16.242496 4822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4280629655dd4a24847f899af71592a4" candidate_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f374783c869641afbd63a8d0027467b1" is_pre_election: true
I20260928 16:00:16.243124 4822 raft_consensus.cc:2454] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe73d580f8ed4ea2bb4af9585b43d79e in term 0.
I20260928 16:00:16.243417 4687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4280629655dd4a24847f899af71592a4" candidate_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" is_pre_election: true
I20260928 16:00:16.243896 4891 leader_election.cc:302] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f374783c869641afbd63a8d0027467b1, fe73d580f8ed4ea2bb4af9585b43d79e; no voters:
I20260928 16:00:16.244046 4687 raft_consensus.cc:2454] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe73d580f8ed4ea2bb4af9585b43d79e in term 0.
I20260928 16:00:16.244342 5025 raft_consensus.cc:2781] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:00:16.244493 5025 raft_consensus.cc:482] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:00:16.244601 5025 raft_consensus.cc:3037] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Advancing to term 1
W20260928 16:00:16.245834 5002 tablet.cc:2369] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:16.247313 5025 raft_consensus.cc:504] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:16.247835 5025 leader_election.cc:288] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [CANDIDATE]: Term 1 election: Requested vote from peers f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701), 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197)
I20260928 16:00:16.248329 4822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4280629655dd4a24847f899af71592a4" candidate_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f374783c869641afbd63a8d0027467b1"
I20260928 16:00:16.248368 4687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4280629655dd4a24847f899af71592a4" candidate_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e"
I20260928 16:00:16.248548 4822 raft_consensus.cc:3037] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:16.248548 4687 raft_consensus.cc:3037] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:16.251757 4822 raft_consensus.cc:2454] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe73d580f8ed4ea2bb4af9585b43d79e in term 1.
I20260928 16:00:16.251757 4687 raft_consensus.cc:2454] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe73d580f8ed4ea2bb4af9585b43d79e in term 1.
I20260928 16:00:16.252207 4890 leader_election.cc:302] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e, fe73d580f8ed4ea2bb4af9585b43d79e; no voters:
I20260928 16:00:16.252593 5025 raft_consensus.cc:2781] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:00:16.253080 5025 raft_consensus.cc:686] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 LEADER]: Becoming Leader. State: Replica: fe73d580f8ed4ea2bb4af9585b43d79e, State: Running, Role: LEADER
I20260928 16:00:16.253614 5025 consensus_queue.cc:237] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
W20260928 16:00:16.258773 4734 tablet.cc:2369] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:16.258592 4540 catalog_manager.cc:5495] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e reported cstate change: term changed from 0 to 1, leader changed from <none> to fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195). New cstate: current_term: 1 leader_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } health_report { overall_health: UNKNOWN } } }
W20260928 16:00:16.266137 4869 tablet.cc:2369] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:16.315006 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.309770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.22.125.195:36647)
0928 16:00:16.309947 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.309952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.309968 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:00:16.310115 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.310119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.310200 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.310334 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:00:16.310642 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.311339 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.312986 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.313533 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.313627 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.314082 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:00:16.314564 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:00:16.314676 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.314752 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.314816 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 16:00:16.323194 4687 raft_consensus.cc:1261] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Refusing update from remote peer fe73d580f8ed4ea2bb4af9585b43d79e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:16.323194 4822 raft_consensus.cc:1261] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 1 FOLLOWER]: Refusing update from remote peer fe73d580f8ed4ea2bb4af9585b43d79e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:16.324354 5029 consensus_queue.cc:1035] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:16.324710 5025 consensus_queue.cc:1035] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:16.336723 5033 mvcc.cc:205] Tried to move back new op lower bound from 7334343542048817152 to 7334343541779214336. Current Snapshot: MvccSnapshot[applied={T|T < 7334343542048817152}]
I20260928 16:00:16.340852 5034 mvcc.cc:205] Tried to move back new op lower bound from 7334343542048817152 to 7334343541779214336. Current Snapshot: MvccSnapshot[applied={T|T < 7334343542048817152}]
I20260928 16:00:16.349714 4540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:16.349992 4540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:16.353677 4540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Sending DeleteTablet for 3 replicas of tablet 4280629655dd4a24847f899af71592a4
I20260928 16:00:16.354779 4802 tablet_service.cc:1514] Processing DeleteTablet for tablet 4280629655dd4a24847f899af71592a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:16 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:35892
I20260928 16:00:16.354884 4667 tablet_service.cc:1514] Processing DeleteTablet for tablet 4280629655dd4a24847f899af71592a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:16 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:58354
I20260928 16:00:16.355019 4936 tablet_service.cc:1514] Processing DeleteTablet for tablet 4280629655dd4a24847f899af71592a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:16 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:54698
I20260928 16:00:16.355854 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4603
I20260928 16:00:16.383749 5041 tablet_replica.cc:330] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: stopping tablet replica
I20260928 16:00:16.384405 5043 tablet_replica.cc:330] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: stopping tablet replica
I20260928 16:00:16.384855 5043 raft_consensus.cc:2229] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:00:16.385499 5043 raft_consensus.cc:2258] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:16.386211 5041 raft_consensus.cc:2229] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:16.386662 5041 raft_consensus.cc:2258] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:16.388563 5042 tablet_replica.cc:330] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: stopping tablet replica
I20260928 16:00:16.388962 5042 raft_consensus.cc:2229] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:16.389431 5042 raft_consensus.cc:2258] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:16.391806 5042 ts_tablet_manager.cc:1905] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:16.391805 5041 ts_tablet_manager.cc:1905] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:00:16 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 16:00:16.398881 5041 ts_tablet_manager.cc:1918] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:16.398941 5042 ts_tablet_manager.cc:1918] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:16.399044 5041 log.cc:1196] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/4280629655dd4a24847f899af71592a4
I20260928 16:00:16.399082 5042 log.cc:1196] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/4280629655dd4a24847f899af71592a4
I20260928 16:00:16.399641 5042 ts_tablet_manager.cc:1939] T 4280629655dd4a24847f899af71592a4 P 38a11e64b7214e35b5dcab8d8ff9a03e: Deleting consensus metadata
I20260928 16:00:16.401513 5043 ts_tablet_manager.cc:1905] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:16.401633 4526 catalog_manager.cc:4841] TS 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197): tablet 4280629655dd4a24847f899af71592a4 (table test-table [id=02c3c2e110e346a3a795875097294ce6]) successfully deleted
I20260928 16:00:16.404558 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.357244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.22.125.193:44197)
0928 16:00:16.357478 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.357487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.379724 (+ 22237us) server_negotiation.cc:407] Connection header received
0928 16:00:16.380319 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.380328 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.380454 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.380548 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:16.387122 (+ 6574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.387952 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.390078 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.390763 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.390920 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.396664 (+ 5744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:16.396678 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:16.396700 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:16.396766 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:16.400834 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.401973 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.401981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.401984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.402148 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.402622 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.402627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.402631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.403079 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:16.403398 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.403726 (+ 328us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.403952 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:00:16.405221 4728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:16.409474 5043 ts_tablet_manager.cc:1918] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:16.409644 5043 log.cc:1196] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/4280629655dd4a24847f899af71592a4
I20260928 16:00:16.410197 5043 ts_tablet_manager.cc:1939] T 4280629655dd4a24847f899af71592a4 P fe73d580f8ed4ea2bb4af9585b43d79e: Deleting consensus metadata
I20260928 16:00:16.412007 5041 ts_tablet_manager.cc:1939] T 4280629655dd4a24847f899af71592a4 P f374783c869641afbd63a8d0027467b1: Deleting consensus metadata
I20260928 16:00:16.412025 4528 catalog_manager.cc:4841] TS fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647): tablet 4280629655dd4a24847f899af71592a4 (table test-table [id=02c3c2e110e346a3a795875097294ce6]) successfully deleted
I20260928 16:00:16.413808 4527 catalog_manager.cc:4841] TS f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701): tablet 4280629655dd4a24847f899af71592a4 (table test-table [id=02c3c2e110e346a3a795875097294ce6]) successfully deleted
I20260928 16:00:16.509953 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4603
I20260928 16:00:16.532833 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4739
Sep 28 16:00:16 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 16:00:16.550065 5015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.533669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.22.125.194:41701)
0928 16:00:16.533861 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.533868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.534044 (+ 176us) server_negotiation.cc:407] Connection header received
0928 16:00:16.534253 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.534259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.534345 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.534457 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:16.535908 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.536819 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.538108 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.538528 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.538674 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.543200 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:16.543210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:16.543224 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:16.543263 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:16.546598 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.547438 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.547445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.547447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.547583 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.548007 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.548014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.548017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.548452 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:16.548774 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.549638 (+ 864us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.549807 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:00:16.550755 4863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:16.641805 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4739
I20260928 16:00:16.659876 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4873
Sep 28 16:00:16 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 16:00:16.677067 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:16.660738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.22.125.195:36647)
0928 16:00:16.660952 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:16.660958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:16.660982 (+ 24us) server_negotiation.cc:407] Connection header received
0928 16:00:16.661053 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:16.661058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:16.661154 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:16.661293 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:16.662547 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.663815 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.665259 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:16.665961 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:16.666129 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:16.670576 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:16.670592 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:16.670613 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:16.670688 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:16.674192 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.675000 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.675007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.675009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.675162 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:16.675597 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:16.675600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:16.675602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:16.675932 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:16.676236 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:16.676555 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 16:00:16.676707 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260928 16:00:16.677805 4996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:16.804839 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4873
I20260928 16:00:16.825157 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 4509
I20260928 16:00:16.826295 4572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:16.939299 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 4509
I20260928 16:00:16.960287 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39405
--webserver_interface=127.22.125.254
--webserver_port=39791
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:39405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:17.287678 5049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:17.288031 5049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:17.288218 5049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:17.295347 5049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:17.295485 5049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:17.295593 5049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:17.295699 5049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:17.295786 5049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:17.305382 5049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:39405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=39791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:17.306694 5049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:17.308097 5049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:17.330978 5055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:17.330928 5058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:17.331014 5056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:17.332088 5049 server_base.cc:1034] running on GCE node
I20260928 16:00:17.333031 5049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:17.334596 5049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:17.335834 5049 hybrid_clock.cc:648] HybridClock initialized: now 1790611217335770 us; error 51 us; skew 500 ppm
I20260928 16:00:17.336320 5049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:17 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611217, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:17.339816 5049 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:00:17.341655 5049 webserver.cc:460] Webserver started at http://127.22.125.254:39791/ using document root <none> and password file <none>
I20260928 16:00:17.342422 5049 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:17.342592 5049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:17.348704 5049 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260928 16:00:17.351540 5065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:17.352891 5049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:00:17.353086 5049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "d317f135e8f4417c9cff8e691c2f296b"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "c20080dc2e21faf65a646673e62ecc1e"
server_key_iv: "4da342ddd5325af35fdf96267f3938b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:17.354141 5049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:17.366043 5049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:17.370215 5049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:17.370501 5049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:17.389593 5049 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:39405
I20260928 16:00:17.389617 5117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:39405 every 8 connection(s)
I20260928 16:00:17.391134 5049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:00:17.392385 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5049
I20260928 16:00:17.393446 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:44197
--local_ip_for_outbound_sockets=127.22.125.193
--tserver_master_addrs=127.22.125.254:39405
--webserver_port=45433
--webserver_interface=127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:17.401465 5118 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:00:17.404893 5118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Bootstrap starting.
I20260928 16:00:17.424692 5118 log.cc:824] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:17.436668 5118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:00:17.437207 5118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Bootstrap complete.
I20260928 16:00:17.446768 5118 raft_consensus.cc:348] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:17.447623 5118 raft_consensus.cc:729] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d317f135e8f4417c9cff8e691c2f296b, State: Initialized, Role: FOLLOWER
I20260928 16:00:17.448304 5118 consensus_queue.cc:260] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:17.448559 5118 raft_consensus.cc:388] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:00:17.448688 5118 raft_consensus.cc:482] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:00:17.448886 5118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:00:17.452787 5118 raft_consensus.cc:504] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:17.453311 5118 leader_election.cc:302] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d317f135e8f4417c9cff8e691c2f296b; no voters:
I20260928 16:00:17.454000 5118 leader_election.cc:288] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 16:00:17.454133 5122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:00:17.455407 5122 raft_consensus.cc:686] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 LEADER]: Becoming Leader. State: Replica: d317f135e8f4417c9cff8e691c2f296b, State: Running, Role: LEADER
I20260928 16:00:17.456001 5122 consensus_queue.cc:237] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:17.456374 5118 sys_catalog.cc:564] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:17.458623 5124 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d317f135e8f4417c9cff8e691c2f296b. Latest consensus state: current_term: 2 leader_uuid: "d317f135e8f4417c9cff8e691c2f296b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } } }
I20260928 16:00:17.458992 5124 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:17.458796 5123 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d317f135e8f4417c9cff8e691c2f296b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } } }
I20260928 16:00:17.459208 5123 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:17.460608 5128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:00:17.475713 5128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:00:17.477219 5128 catalog_manager.cc:1259] Loaded cluster ID: 3538ffcd61bb43fc9cc9a3a108da333f
I20260928 16:00:17.477371 5128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:00:17.480531 5128 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 16:00:17.498890 5128 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 16:00:17.524433 5128 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:00:17.526310 5128 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Loaded TSK: 0
I20260928 16:00:17.527908 5128 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:00:17.752667 5120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:17.752918 5120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:17.753053 5120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:17.760265 5120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:17.760358 5120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:17.760528 5120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:00:17.769934 5120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:44197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=45433
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:17.771240 5120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:17.772593 5120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:17.786165 5145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:17.786092 5146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:17.786427 5148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:17.793617 5120 server_base.cc:1034] running on GCE node
I20260928 16:00:17.794214 5120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:17.795203 5120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:17.796416 5120 hybrid_clock.cc:648] HybridClock initialized: now 1790611217796363 us; error 48 us; skew 500 ppm
I20260928 16:00:17.796732 5120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:17 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611217, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:17.800010 5120 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:00:17.801539 5120 webserver.cc:460] Webserver started at http://127.22.125.193:45433/ using document root <none> and password file <none>
I20260928 16:00:17.802258 5120 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:17.802417 5120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:17.807870 5120 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:00:17.810711 5155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:17.812222 5120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:00:17.812397 5120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "38a11e64b7214e35b5dcab8d8ff9a03e"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "a0444bd797d7ae4f125070c646f452c4"
server_key_iv: "3eeaddbdcaa8020118bcbad0de98b4eb"
server_key_version: "encryptionkey@0"
I20260928 16:00:17.813369 5120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:17.841101 5120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:17.845475 5120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:17.845757 5120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:17.846745 5120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:17.848206 5120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:17.848325 5120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:17.848465 5120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:17.848547 5120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:17.881897 5120 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:44197
I20260928 16:00:17.882011 5268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:44197 every 8 connection(s)
I20260928 16:00:17.883394 5120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:00:17.889811 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5120
I20260928 16:00:17.890573 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:41701
--local_ip_for_outbound_sockets=127.22.125.194
--tserver_master_addrs=127.22.125.254:39405
--webserver_port=40557
--webserver_interface=127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:00:17 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611217, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:17.908927 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:17.886300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:49245 (local address 127.22.125.254:39405)
0928 16:00:17.886879 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:17.886891 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:17.887440 (+ 549us) server_negotiation.cc:407] Connection header received
0928 16:00:17.889563 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:17.889575 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:17.889930 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:17.890241 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:17.892372 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:17.893570 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:17.896125 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:17.896737 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:17.896878 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:17.901550 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:17.901558 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:17.901571 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:17.901605 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:17.904463 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:17.905427 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:17.905433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:17.905438 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:17.905599 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:17.906094 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:17.906097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:17.906112 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:17.906553 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:17.906694 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:17.907631 (+ 937us) server_negotiation.cc:299] Negotiation successful
0928 16:00:17.907922 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":255,"threads_started":1}
I20260928 16:00:17.909441 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:17.886467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:49245)
0928 16:00:17.887270 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:17.887322 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:17.889063 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:17.890456 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:17.890469 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:17.891286 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:17.892144 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:17.892171 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:17.893750 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:17.893759 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:17.895935 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:17.895953 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:17.896981 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:17.896991 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:17.897309 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:17.898041 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:17.898076 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:17.901267 (+ 3191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:17.904612 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:17.904622 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:17.904640 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:17.905289 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:17.905714 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:17.905722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:17.905728 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:17.905983 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:17.906702 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:17.906712 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:17.907150 (+ 438us) client_negotiation.cc:769] Sending connection context
0928 16:00:17.907444 (+ 294us) client_negotiation.cc:240] Negotiation successful
0928 16:00:17.908022 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":339,"threads_started":1}
I20260928 16:00:17.912225 5269 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39405
I20260928 16:00:17.912534 5269 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:17.913237 5269 heartbeater.cc:511] Master 127.22.125.254:39405 requested a full tablet report, sending...
I20260928 16:00:17.915367 5082 ts_manager.cc:194] Registered new tserver with Master: 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197)
I20260928 16:00:17.916952 5082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:49245
W20260928 16:00:18.214501 5273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:18.214829 5273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:18.215010 5273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:18.222477 5273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:18.222625 5273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:18.222808 5273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:00:18.232348 5273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:41701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=40557
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:18.233678 5273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:18.235000 5273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:18.247769 5279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:18.247807 5278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:18.247784 5281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:18.248714 5273 server_base.cc:1034] running on GCE node
I20260928 16:00:18.249346 5273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:18.250259 5273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:18.251485 5273 hybrid_clock.cc:648] HybridClock initialized: now 1790611218251439 us; error 40 us; skew 500 ppm
I20260928 16:00:18.251792 5273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:18 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611218, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:18.254927 5273 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:00:18.256335 5273 webserver.cc:460] Webserver started at http://127.22.125.194:40557/ using document root <none> and password file <none>
I20260928 16:00:18.257061 5273 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:18.257218 5273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:18.263439 5273 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260928 16:00:18.266078 5288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:18.267390 5273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:00:18.267562 5273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "f374783c869641afbd63a8d0027467b1"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "63cfac90035cd463dccac1d8495b8b98"
server_key_iv: "3d8aff717bc93b0e52e870bf68c43c74"
server_key_version: "encryptionkey@0"
I20260928 16:00:18.268567 5273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:18.293635 5273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:18.297724 5273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:18.298017 5273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:18.299018 5273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:18.300326 5273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:18.300478 5273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:18.300616 5273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:18.300714 5273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:18.333619 5273 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:41701
I20260928 16:00:18.333705 5401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:41701 every 8 connection(s)
I20260928 16:00:18.335098 5273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:00:18.335675 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5273
I20260928 16:00:18.336541 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:36647
--local_ip_for_outbound_sockets=127.22.125.195
--tserver_master_addrs=127.22.125.254:39405
--webserver_port=37237
--webserver_interface=127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:00:18 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611218, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:18.357283 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.339117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:59557 (local address 127.22.125.254:39405)
0928 16:00:18.339333 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:18.339340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:18.339596 (+ 256us) server_negotiation.cc:407] Connection header received
0928 16:00:18.341539 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:18.341545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:18.341639 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:18.341754 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:18.343229 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.343939 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.345508 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.345939 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.346078 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.349916 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:18.349925 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:18.349933 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:18.349979 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:18.353629 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:18.354411 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:18.354418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:18.354425 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:18.354592 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:18.355028 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:18.355036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:18.355041 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:18.355461 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:18.355632 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:18.356430 (+ 798us) server_negotiation.cc:299] Negotiation successful
0928 16:00:18.356711 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:00:18.357868 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.338626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:59557)
0928 16:00:18.339449 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:18.339490 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:18.341264 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:18.341997 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:18.342005 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:18.342499 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:18.343054 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.343071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.344129 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.344134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.345356 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.345368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.346178 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.346185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.346421 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.347110 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:18.347132 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:18.349681 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:18.353810 (+ 4129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:18.353820 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:18.353839 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:18.354270 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:18.354716 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:18.354719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:18.354721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:18.354902 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:18.355601 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:18.355606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:18.355918 (+ 312us) client_negotiation.cc:769] Sending connection context
0928 16:00:18.356250 (+ 332us) client_negotiation.cc:240] Negotiation successful
0928 16:00:18.356641 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":453,"threads_started":1}
I20260928 16:00:18.359436 5402 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39405
I20260928 16:00:18.359745 5402 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:18.360577 5402 heartbeater.cc:511] Master 127.22.125.254:39405 requested a full tablet report, sending...
I20260928 16:00:18.362052 5082 ts_manager.cc:194] Registered new tserver with Master: f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701)
I20260928 16:00:18.362852 5082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:59557
W20260928 16:00:18.671370 5405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:18.671659 5405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:18.671844 5405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:18.679134 5405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:18.679239 5405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:18.679410 5405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:00:18.689337 5405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:36647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=37237
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:18.690651 5405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:18.691975 5405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:18.705425 5411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:18.705390 5413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:18.705458 5410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:18.706120 5405 server_base.cc:1034] running on GCE node
I20260928 16:00:18.706712 5405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:18.707602 5405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:18.708792 5405 hybrid_clock.cc:648] HybridClock initialized: now 1790611218708745 us; error 37 us; skew 500 ppm
I20260928 16:00:18.709107 5405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:18 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611218, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:18.712203 5405 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:00:18.713510 5405 webserver.cc:460] Webserver started at http://127.22.125.195:37237/ using document root <none> and password file <none>
I20260928 16:00:18.714154 5405 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:18.714308 5405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:18.719560 5405 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:00:18.722138 5420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:18.723392 5405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:00:18.723555 5405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "fe73d580f8ed4ea2bb4af9585b43d79e"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "116dc073588944550895b5b7b414b2a3"
server_key_iv: "72fe64f92a9eaebe4359b751ccd1d71d"
server_key_version: "encryptionkey@0"
I20260928 16:00:18.724591 5405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:18.742460 5405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:18.746820 5405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:18.747100 5405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:18.748088 5405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:18.749465 5405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:18.749550 5405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:18.749711 5405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:18.749794 5405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:18.782884 5405 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:36647
I20260928 16:00:18.782977 5534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:36647 every 8 connection(s)
I20260928 16:00:18.784345 5405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:00:18.791484 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5405
Sep 28 16:00:18 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611218, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:18.804176 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.787057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:52303 (local address 127.22.125.254:39405)
0928 16:00:18.787278 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:18.787284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:18.788449 (+ 1165us) server_negotiation.cc:407] Connection header received
0928 16:00:18.790137 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:18.790141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:18.790208 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:18.790295 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:18.791767 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.792798 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.794480 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.794897 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.795034 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.798506 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:18.798511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:18.798514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:18.798544 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:18.801111 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:18.801861 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:18.801867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:18.801870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:18.802007 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:18.802441 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:18.802447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:18.802450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:18.802713 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:18.802892 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:18.803562 (+ 670us) server_negotiation.cc:299] Negotiation successful
0928 16:00:18.803778 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"spinlock_wait_cycles":7552}
I20260928 16:00:18.805444 5537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.787566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:52303)
0928 16:00:18.788300 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:18.788339 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:18.789897 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:18.790497 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:18.790504 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:18.791019 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:18.791564 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.791581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.793045 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.793054 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.794325 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.794337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.795150 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.795160 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.795416 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.796131 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:18.796158 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:18.798287 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:18.801299 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:18.801307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:18.801322 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:18.801725 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:18.802128 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:18.802134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:18.802138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:18.802319 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:18.802852 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:18.802859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:18.803190 (+ 331us) client_negotiation.cc:769] Sending connection context
0928 16:00:18.803470 (+ 280us) client_negotiation.cc:240] Negotiation successful
0928 16:00:18.803982 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":283,"threads_started":1}
I20260928 16:00:18.806561 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.792584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.22.125.254:39405)
0928 16:00:18.793419 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:18.793423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:18.793436 (+ 13us) server_negotiation.cc:407] Connection header received
0928 16:00:18.793471 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:18.793475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:18.793526 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:18.793637 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:18.795057 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.795951 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.797398 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.797897 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.798023 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.800989 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:18.800999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:18.801001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:18.801044 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:18.803670 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:18.804494 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:18.804500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:18.804504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:18.804642 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:18.805003 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:18.805008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:18.805011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:18.805348 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:18.805605 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:18.805953 (+ 348us) server_negotiation.cc:299] Negotiation successful
0928 16:00:18.806191 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":551,"threads_started":1}
I20260928 16:00:18.806990 5535 heartbeater.cc:347] Connected to a master server at 127.22.125.254:39405
I20260928 16:00:18.807334 5535 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:18.808282 5535 heartbeater.cc:511] Master 127.22.125.254:39405 requested a full tablet report, sending...
I20260928 16:00:18.809832 5082 ts_manager.cc:194] Registered new tserver with Master: fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647)
I20260928 16:00:18.810570 5082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:52303
I20260928 16:00:18.821630 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:00:18.827945 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.822434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.22.125.254:39405)
0928 16:00:18.822664 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:18.822673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:18.823040 (+ 367us) server_negotiation.cc:407] Connection header received
0928 16:00:18.823198 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:18.823202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:18.823349 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:18.823468 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:00:18.823845 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.824598 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.825955 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.826386 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.826475 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.826888 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:00:18.827401 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:00:18.827517 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:18.827600 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 16:00:18.827661 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:00:18.828871 5082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:00:18.836557 5082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:00:18.875134 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.848605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35910 (local address 127.22.125.194:41701)
0928 16:00:18.859118 (+ 10513us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:18.859129 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:18.865321 (+ 6192us) server_negotiation.cc:407] Connection header received
0928 16:00:18.865896 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:18.865910 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:18.866278 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:18.866481 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:18.867702 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.869282 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.872183 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.873653 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.873810 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.873996 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:18.874611 (+ 615us) server_negotiation.cc:299] Negotiation successful
0928 16:00:18.874797 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10307,"thread_start_us":10198,"threads_started":1}
I20260928 16:00:18.875167 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.848211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:41701 (local address 127.0.0.1:35910)
0928 16:00:18.865164 (+ 16953us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:18.865189 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:18.865641 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:18.866938 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:18.866948 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:18.866984 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:18.867432 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.867452 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.869460 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.869467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.871919 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.871998 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.874151 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.874157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.874362 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.874417 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 16:00:18.874551 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 16:00:18.874721 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16776,"thread_start_us":12609,"threads_started":1}
I20260928 16:00:18.876592 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.850544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58382)
0928 16:00:18.867777 (+ 17233us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:18.867797 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:18.867905 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:18.868989 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:18.868994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:18.869016 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:18.869349 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.869361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.871224 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.871229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.873426 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.873442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.875787 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.875791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.875927 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.875949 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 16:00:18.876085 (+ 136us) client_negotiation.cc:240] Negotiation successful
0928 16:00:18.876211 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17099,"mutex_wait_us":2,"thread_start_us":16959,"threads_started":1}
I20260928 16:00:18.876645 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.850772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.22.125.193:44197)
0928 16:00:18.867292 (+ 16520us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:18.867302 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:18.868237 (+ 935us) server_negotiation.cc:407] Connection header received
0928 16:00:18.868347 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:18.868354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:18.868568 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:18.868757 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:18.869540 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.871055 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.873613 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.875020 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.875202 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.875596 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:18.876125 (+ 529us) server_negotiation.cc:299] Negotiation successful
0928 16:00:18.876288 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16343,"thread_start_us":17198,"threads_started":1}
I20260928 16:00:18.879168 5337 tablet_service.cc:1467] Processing CreateTablet for tablet 9287a971b89c4e8e976d6e96aa073d8c (DEFAULT_TABLE table=test-table [id=1f9bd486a86748bb89af296b78720f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:18.879936 5204 tablet_service.cc:1467] Processing CreateTablet for tablet 9287a971b89c4e8e976d6e96aa073d8c (DEFAULT_TABLE table=test-table [id=1f9bd486a86748bb89af296b78720f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:18.880708 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.847933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:36647 (local address 127.0.0.1:54728)
0928 16:00:18.861198 (+ 13265us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:18.861243 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:18.861414 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:18.871920 (+ 10506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:18.871925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:18.872009 (+ 84us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:18.872518 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.872531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.874903 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.874908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.878196 (+ 3288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:18.878210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.879922 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.879930 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:18.880144 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.880177 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:00:18.880265 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 16:00:18.880384 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13031,"thread_start_us":17423,"threads_started":1}
I20260928 16:00:18.880789 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9287a971b89c4e8e976d6e96aa073d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:18.880874 5547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:18.848214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.22.125.195:36647)
0928 16:00:18.871472 (+ 23258us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:18.871478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:18.871505 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:00:18.871572 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:18.871576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:18.871721 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:18.871920 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:18.872725 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.874704 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.878369 (+ 3665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:18.879685 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:18.879892 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:18.880083 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:18.880353 (+ 270us) server_negotiation.cc:299] Negotiation successful
0928 16:00:18.880530 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23093,"thread_start_us":22942,"threads_started":1}
I20260928 16:00:18.881839 5204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9287a971b89c4e8e976d6e96aa073d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:18.884321 5469 tablet_service.cc:1467] Processing CreateTablet for tablet 9287a971b89c4e8e976d6e96aa073d8c (DEFAULT_TABLE table=test-table [id=1f9bd486a86748bb89af296b78720f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:18.885584 5469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9287a971b89c4e8e976d6e96aa073d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:18.897526 5551 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Bootstrap starting.
I20260928 16:00:18.899242 5552 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Bootstrap starting.
I20260928 16:00:18.901352 5551 tablet_bootstrap.cc:654] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:18.902354 5551 log.cc:824] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:18.903275 5553 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Bootstrap starting.
I20260928 16:00:18.903470 5552 tablet_bootstrap.cc:654] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:18.904906 5552 log.cc:824] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:18.906842 5553 tablet_bootstrap.cc:654] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:18.906944 5551 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: No bootstrap required, opened a new log
I20260928 16:00:18.907266 5551 ts_tablet_manager.cc:1397] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260928 16:00:18.907578 5552 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: No bootstrap required, opened a new log
I20260928 16:00:18.907836 5552 ts_tablet_manager.cc:1397] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260928 16:00:18.908025 5553 log.cc:824] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:18.910955 5553 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: No bootstrap required, opened a new log
I20260928 16:00:18.911206 5553 ts_tablet_manager.cc:1397] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260928 16:00:18.915297 5552 raft_consensus.cc:348] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:18.915854 5552 raft_consensus.cc:374] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:18.916010 5552 raft_consensus.cc:729] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a11e64b7214e35b5dcab8d8ff9a03e, State: Initialized, Role: FOLLOWER
I20260928 16:00:18.916798 5552 consensus_queue.cc:260] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:18.917531 5551 raft_consensus.cc:348] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:18.917922 5551 raft_consensus.cc:374] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:18.918015 5551 raft_consensus.cc:729] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f374783c869641afbd63a8d0027467b1, State: Initialized, Role: FOLLOWER
I20260928 16:00:18.918571 5551 consensus_queue.cc:260] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:18.919754 5402 heartbeater.cc:503] Master 127.22.125.254:39405 was elected leader, sending a full tablet report...
I20260928 16:00:18.919881 5269 heartbeater.cc:503] Master 127.22.125.254:39405 was elected leader, sending a full tablet report...
I20260928 16:00:18.921458 5553 raft_consensus.cc:348] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:18.921849 5553 raft_consensus.cc:374] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:18.921954 5553 raft_consensus.cc:729] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe73d580f8ed4ea2bb4af9585b43d79e, State: Initialized, Role: FOLLOWER
I20260928 16:00:18.922468 5553 consensus_queue.cc:260] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:18.923647 5535 heartbeater.cc:503] Master 127.22.125.254:39405 was elected leader, sending a full tablet report...
I20260928 16:00:18.924299 5553 ts_tablet_manager.cc:1428] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260928 16:00:18.925868 5551 ts_tablet_manager.cc:1428] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Time spent starting tablet: real 0.018s user 0.013s sys 0.000s
I20260928 16:00:18.926447 5552 ts_tablet_manager.cc:1428] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260928 16:00:19.007131 5558 raft_consensus.cc:482] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:19.007403 5558 raft_consensus.cc:504] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:19.009059 5558 leader_election.cc:288] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647), 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197)
I20260928 16:00:19.015453 5547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.009467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:53753 (local address 127.22.125.195:36647)
0928 16:00:19.009904 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:19.009913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:19.010263 (+ 350us) server_negotiation.cc:407] Connection header received
0928 16:00:19.010453 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:19.010458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:19.010527 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:19.010641 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:19.011195 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.011927 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.013739 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.014516 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.014706 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.014782 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:19.015079 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 16:00:19.015199 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260928 16:00:19.015527 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.009603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:38591 (local address 127.22.125.193:44197)
0928 16:00:19.010104 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:19.010110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:19.010491 (+ 381us) server_negotiation.cc:407] Connection header received
0928 16:00:19.010682 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:19.010686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:19.010745 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:19.010852 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:19.011413 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.012181 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.013876 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.014614 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.014784 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.014872 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:19.015150 (+ 278us) server_negotiation.cc:299] Negotiation successful
0928 16:00:19.015273 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260928 16:00:19.015544 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.009254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:36647 (local address 127.22.125.194:53753)
0928 16:00:19.010130 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.010154 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:19.010318 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:19.010663 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:19.010667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:19.010696 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:19.011052 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:19.011060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.012084 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.012090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:19.013577 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:19.013589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.014736 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.014742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:19.014914 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.014937 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 16:00:19.015042 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 16:00:19.015173 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":485,"threads_started":1}
I20260928 16:00:19.015588 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.009366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.22.125.194:38591)
0928 16:00:19.010397 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.010414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:19.010521 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:19.010868 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:19.010873 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:19.010896 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:19.011260 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:19.011268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.012328 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.012332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:19.013715 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:19.013727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.014833 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.014840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:19.014996 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.015018 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 16:00:19.015121 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 16:00:19.015238 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":693,"threads_started":1}
I20260928 16:00:19.016557 5224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9287a971b89c4e8e976d6e96aa073d8c" candidate_uuid: "f374783c869641afbd63a8d0027467b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" is_pre_election: true
I20260928 16:00:19.016760 5489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9287a971b89c4e8e976d6e96aa073d8c" candidate_uuid: "f374783c869641afbd63a8d0027467b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" is_pre_election: true
I20260928 16:00:19.017167 5224 raft_consensus.cc:2454] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f374783c869641afbd63a8d0027467b1 in term 0.
I20260928 16:00:19.017264 5489 raft_consensus.cc:2454] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f374783c869641afbd63a8d0027467b1 in term 0.
I20260928 16:00:19.017892 5290 leader_election.cc:302] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e, f374783c869641afbd63a8d0027467b1; no voters:
I20260928 16:00:19.018328 5558 raft_consensus.cc:2781] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:00:19.018501 5558 raft_consensus.cc:482] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:00:19.018606 5558 raft_consensus.cc:3037] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:19.020931 5558 raft_consensus.cc:504] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:19.021490 5558 leader_election.cc:288] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [CANDIDATE]: Term 1 election: Requested vote from peers fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647), 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197)
I20260928 16:00:19.021903 5224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9287a971b89c4e8e976d6e96aa073d8c" candidate_uuid: "f374783c869641afbd63a8d0027467b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e"
I20260928 16:00:19.021942 5489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9287a971b89c4e8e976d6e96aa073d8c" candidate_uuid: "f374783c869641afbd63a8d0027467b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e"
I20260928 16:00:19.022080 5224 raft_consensus.cc:3037] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:19.022163 5489 raft_consensus.cc:3037] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:19.024290 5224 raft_consensus.cc:2454] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f374783c869641afbd63a8d0027467b1 in term 1.
I20260928 16:00:19.024482 5489 raft_consensus.cc:2454] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f374783c869641afbd63a8d0027467b1 in term 1.
I20260928 16:00:19.024711 5290 leader_election.cc:302] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e, f374783c869641afbd63a8d0027467b1; no voters:
I20260928 16:00:19.024986 5558 raft_consensus.cc:2781] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:00:19.025386 5558 raft_consensus.cc:686] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 1 LEADER]: Becoming Leader. State: Replica: f374783c869641afbd63a8d0027467b1, State: Running, Role: LEADER
I20260928 16:00:19.025810 5558 consensus_queue.cc:237] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:19.030459 5081 catalog_manager.cc:5495] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f374783c869641afbd63a8d0027467b1 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "f374783c869641afbd63a8d0027467b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } health_report { overall_health: UNKNOWN } } }
W20260928 16:00:19.035992 5536 tablet.cc:2369] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:19.067859 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.062860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.22.125.194:41701)
0928 16:00:19.063072 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:19.063077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:19.063095 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:00:19.063243 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:19.063248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:19.063332 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:19.063464 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:00:19.063776 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.064511 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.065843 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.066238 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.066326 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.066781 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:00:19.067272 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:00:19.067391 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:19.067467 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 16:00:19.067530 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:00:19.076131 5489 raft_consensus.cc:1261] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 FOLLOWER]: Refusing update from remote peer f374783c869641afbd63a8d0027467b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:19.076143 5224 raft_consensus.cc:1261] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Refusing update from remote peer f374783c869641afbd63a8d0027467b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:19.077072 5562 consensus_queue.cc:1035] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:19.077450 5558 consensus_queue.cc:1035] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 16:00:19.087230 5403 tablet.cc:2369] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:19.088719 5565 mvcc.cc:205] Tried to move back new op lower bound from 7334343553326186496 to 7334343553133801472. Current Snapshot: MvccSnapshot[applied={T|T < 7334343553326186496}]
I20260928 16:00:19.092321 5566 mvcc.cc:205] Tried to move back new op lower bound from 7334343553326186496 to 7334343553133801472. Current Snapshot: MvccSnapshot[applied={T|T < 7334343553326186496}]
I20260928 16:00:19.102650 5081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:19.102934 5081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:19.106297 5081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Sending DeleteTablet for 3 replicas of tablet 9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:19.107501 5469 tablet_service.cc:1514] Processing DeleteTablet for tablet 9287a971b89c4e8e976d6e96aa073d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:19 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:54728
I20260928 16:00:19.107499 5337 tablet_service.cc:1514] Processing DeleteTablet for tablet 9287a971b89c4e8e976d6e96aa073d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:19 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:35910
I20260928 16:00:19.108101 5204 tablet_service.cc:1514] Processing DeleteTablet for tablet 9287a971b89c4e8e976d6e96aa073d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:19 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:58382
I20260928 16:00:19.108494 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 5120
I20260928 16:00:19.122769 5575 tablet_replica.cc:330] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: stopping tablet replica
I20260928 16:00:19.123404 5575 raft_consensus.cc:2229] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:19.124053 5575 raft_consensus.cc:2258] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:19.126518 5575 ts_tablet_manager.cc:1905] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:19.126938 5577 tablet_replica.cc:330] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: stopping tablet replica
I20260928 16:00:19.127985 5577 raft_consensus.cc:2229] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:19.129040 5577 raft_consensus.cc:2258] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:19.131651 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.110251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.22.125.193:44197)
0928 16:00:19.110474 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:19.110483 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:19.110506 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:00:19.110574 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:19.110582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:19.110689 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:19.110805 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:19.113298 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.114834 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.117791 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.118753 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.118891 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.123310 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:19.123327 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:19.123346 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:19.123387 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:19.127500 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:19.128402 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:19.128410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:19.128414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:19.128569 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:19.129032 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:19.129039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:19.129044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:19.129546 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:19.129819 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:19.131037 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0928 16:00:19.131255 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:00:19.132318 5264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:19.132711 5576 tablet_replica.cc:330] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: stopping tablet replica
I20260928 16:00:19.133301 5576 raft_consensus.cc:2229] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:00:19.133667 5575 ts_tablet_manager.cc:1918] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:19.133879 5575 log.cc:1196] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:19.134132 5576 raft_consensus.cc:2258] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:19.134545 5575 ts_tablet_manager.cc:1939] T 9287a971b89c4e8e976d6e96aa073d8c P fe73d580f8ed4ea2bb4af9585b43d79e: Deleting consensus metadata
I20260928 16:00:19.136379 5576 ts_tablet_manager.cc:1905] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:19.136472 5069 catalog_manager.cc:4841] TS fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647): tablet 9287a971b89c4e8e976d6e96aa073d8c (table test-table [id=1f9bd486a86748bb89af296b78720f87]) successfully deleted
I20260928 16:00:19.144881 5576 ts_tablet_manager.cc:1918] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:19.145123 5576 log.cc:1196] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:19.145744 5576 ts_tablet_manager.cc:1939] T 9287a971b89c4e8e976d6e96aa073d8c P f374783c869641afbd63a8d0027467b1: Deleting consensus metadata
I20260928 16:00:19.147641 5068 catalog_manager.cc:4841] TS f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701): tablet 9287a971b89c4e8e976d6e96aa073d8c (table test-table [id=1f9bd486a86748bb89af296b78720f87]) successfully deleted
I20260928 16:00:19.137480 5577 ts_tablet_manager.cc:1905] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:19.228448 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 5120
I20260928 16:00:19.249626 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 5273
W20260928 16:00:19.249563 5067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.125.193:44197 (error 108)
I20260928 16:00:19.251886 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.251407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58394)
0928 16:00:19.251580 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.251712 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 16:00:19.252202 5067 catalog_manager.cc:4584] TS 38a11e64b7214e35b5dcab8d8ff9a03e (127.22.125.193:44197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9287a971b89c4e8e976d6e96aa073d8c: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
I20260928 16:00:19.266937 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.251150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.22.125.194:41701)
0928 16:00:19.251360 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:19.251366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:19.251385 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:00:19.251450 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:19.251455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:19.251533 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:19.251650 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:19.253291 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.254495 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.255793 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.256544 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.256672 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.259493 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:19.259508 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:19.259527 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:19.259566 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:19.263474 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:19.264469 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:19.264478 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:19.264481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:19.264638 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:19.265146 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:19.265153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:19.265157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:19.265713 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:19.266040 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:19.266320 (+ 280us) server_negotiation.cc:299] Negotiation successful
0928 16:00:19.266550 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:00:19.267650 5397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:19.305949 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.305497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58402)
0928 16:00:19.305699 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.305786 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 16:00:19.306947 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.306602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58410)
0928 16:00:19.306750 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.306832 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 16:00:19.367529 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.367051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58426)
0928 16:00:19.367275 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.367363 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260928 16:00:19.368574 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.368226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58428)
0928 16:00:19.368381 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.368463 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 16:00:19.381487 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 5273
I20260928 16:00:19.405334 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 5405
I20260928 16:00:19.425554 5547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.406127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.22.125.195:36647)
0928 16:00:19.406307 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:19.406311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:19.406379 (+ 68us) server_negotiation.cc:407] Connection header received
0928 16:00:19.406537 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:19.406541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:19.406605 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:19.406693 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:19.408611 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.409316 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.411190 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:19.411576 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:19.411665 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:19.419985 (+ 8320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:19.419999 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:19.420014 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:19.420272 (+ 258us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:19.422917 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:19.423639 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:19.423646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:19.423648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:19.423779 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:19.424124 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:19.424127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:19.424129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:19.424464 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:19.424682 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:19.425027 (+ 345us) server_negotiation.cc:299] Negotiation successful
0928 16:00:19.425198 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:00:19.426216 5530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:19.448897 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.448463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58434)
0928 16:00:19.448666 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.448746 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260928 16:00:19.450021 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.449538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58442)
0928 16:00:19.449746 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.449868 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260928 16:00:19.514073 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 5405
I20260928 16:00:19.530784 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 5049
I20260928 16:00:19.531978 5113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:19.646277 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:19.645736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:44197 (local address 127.0.0.1:58448)
0928 16:00:19.645944 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:19.646051 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 16:00:19.646236 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 5049
I20260928 16:00:19.667866 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39405
--webserver_interface=127.22.125.254
--webserver_port=39791
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:39405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:19.990674 5583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:19.991016 5583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:19.991189 5583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:19.998195 5583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:19.998335 5583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:19.998435 5583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:19.998512 5583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:19.998557 5583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:20.007921 5583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:39405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=39791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:20.009315 5583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:20.010675 5583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:20.022403 5589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:20.022374 5590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:20.022451 5592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:20.023248 5583 server_base.cc:1034] running on GCE node
I20260928 16:00:20.024015 5583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:20.025364 5583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:20.026569 5583 hybrid_clock.cc:648] HybridClock initialized: now 1790611220026519 us; error 41 us; skew 500 ppm
I20260928 16:00:20.026925 5583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:20 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:20.030014 5583 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:00:20.031392 5583 webserver.cc:460] Webserver started at http://127.22.125.254:39791/ using document root <none> and password file <none>
I20260928 16:00:20.032137 5583 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:20.032274 5583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:20.037456 5583 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260928 16:00:20.040215 5599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:20.041479 5583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:00:20.041644 5583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "d317f135e8f4417c9cff8e691c2f296b"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "c20080dc2e21faf65a646673e62ecc1e"
server_key_iv: "4da342ddd5325af35fdf96267f3938b2"
server_key_version: "encryptionkey@0"
I20260928 16:00:20.042596 5583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:20.064756 5583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:20.068598 5583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:20.068877 5583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:20.087615 5583 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:39405
I20260928 16:00:20.087637 5650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:39405 every 8 connection(s)
I20260928 16:00:20.089190 5583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:00:20.090176 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5583
I20260928 16:00:20.091284 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:44197
--local_ip_for_outbound_sockets=127.22.125.193
--tserver_master_addrs=127.22.125.254:39405
--webserver_port=45433
--webserver_interface=127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:20.096758 5651 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:00:20.099951 5651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Bootstrap starting.
I20260928 16:00:20.120012 5651 log.cc:824] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:20.136686 5651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:00:20.137235 5651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b: Bootstrap complete.
I20260928 16:00:20.146529 5651 raft_consensus.cc:348] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:20.147644 5651 raft_consensus.cc:729] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d317f135e8f4417c9cff8e691c2f296b, State: Initialized, Role: FOLLOWER
I20260928 16:00:20.148274 5651 consensus_queue.cc:260] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:20.148514 5651 raft_consensus.cc:388] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:00:20.148641 5651 raft_consensus.cc:482] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:00:20.148808 5651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 2 FOLLOWER]: Advancing to term 3
I20260928 16:00:20.152735 5651 raft_consensus.cc:504] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:20.153223 5651 leader_election.cc:302] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d317f135e8f4417c9cff8e691c2f296b; no voters:
I20260928 16:00:20.153846 5651 leader_election.cc:288] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 16:00:20.154038 5655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 3 FOLLOWER]: Leader election won for term 3
I20260928 16:00:20.155251 5655 raft_consensus.cc:686] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [term 3 LEADER]: Becoming Leader. State: Replica: d317f135e8f4417c9cff8e691c2f296b, State: Running, Role: LEADER
I20260928 16:00:20.155934 5651 sys_catalog.cc:564] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:20.155828 5655 consensus_queue.cc:237] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } }
I20260928 16:00:20.160813 5656 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d317f135e8f4417c9cff8e691c2f296b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } } }
I20260928 16:00:20.177048 5656 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:20.161245 5657 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d317f135e8f4417c9cff8e691c2f296b. Latest consensus state: current_term: 3 leader_uuid: "d317f135e8f4417c9cff8e691c2f296b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317f135e8f4417c9cff8e691c2f296b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39405 } } }
I20260928 16:00:20.177626 5657 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317f135e8f4417c9cff8e691c2f296b [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:20.184851 5666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:00:20.193583 5666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:00:20.194977 5666 catalog_manager.cc:1259] Loaded cluster ID: 3538ffcd61bb43fc9cc9a3a108da333f
I20260928 16:00:20.195082 5666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 16:00:20.197570 5666 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fab91c349ad google::LogMessage::Fail() at ??:0
@ 0x7fab91c3788a google::LogMessage::SendToLog() at ??:0
@ 0x7fab91c344de google::LogMessage::Flush() at ??:0
@ 0x7fab91c35a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faba7667512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7faba7664976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faba76ec610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7faba76ec439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fab94880345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fab9488391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fab94883799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fab9485eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fab97b386db start_thread at ??:0
@ 0x7fab90ace61f clone at ??:0
W20260928 16:00:20.520963 5653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:20.521371 5653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:20.521569 5653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:20.532411 5653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:20.532538 5653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:20.532711 5653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:00:20.542557 5653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:44197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=45433
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:20.543824 5653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:20.545195 5653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:20.558090 5693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:20.558135 5694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:20.558051 5696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:20.559721 5653 server_base.cc:1034] running on GCE node
I20260928 16:00:20.560387 5653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:20.561264 5653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:20.562487 5653 hybrid_clock.cc:648] HybridClock initialized: now 1790611220562440 us; error 57 us; skew 500 ppm
I20260928 16:00:20.562796 5653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:20 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:20.565828 5653 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:00:20.567255 5653 webserver.cc:460] Webserver started at http://127.22.125.193:45433/ using document root <none> and password file <none>
I20260928 16:00:20.567929 5653 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:20.568115 5653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:20.573060 5653 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260928 16:00:20.577930 5703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:20.579358 5653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 16:00:20.579536 5653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "38a11e64b7214e35b5dcab8d8ff9a03e"
format_stamp: "Formatted at 2026-09-28 16:00:14 on dist-test-slave-9cqd"
server_key: "a0444bd797d7ae4f125070c646f452c4"
server_key_iv: "3eeaddbdcaa8020118bcbad0de98b4eb"
server_key_version: "encryptionkey@0"
I20260928 16:00:20.580579 5653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:20.597525 5653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:20.601336 5653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:20.601591 5653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:20.602535 5653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:20.604334 5710 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260928 16:00:20.611464 5653 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 16:00:20.611608 5653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260928 16:00:20.611764 5653 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 16:00:20.614919 5653 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 16:00:20.615041 5653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.001s
I20260928 16:00:20.615396 5710 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Bootstrap starting.
I20260928 16:00:20.623694 5710 log.cc:824] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:20.637936 5710 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:00:20.638542 5710 tablet_bootstrap.cc:492] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Bootstrap complete.
I20260928 16:00:20.639006 5710 ts_tablet_manager.cc:1397] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Time spent bootstrapping tablet: real 0.024s user 0.017s sys 0.004s
I20260928 16:00:20.649827 5710 raft_consensus.cc:348] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:20.651074 5710 raft_consensus.cc:729] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a11e64b7214e35b5dcab8d8ff9a03e, State: Initialized, Role: FOLLOWER
I20260928 16:00:20.651937 5710 consensus_queue.cc:260] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:20.653728 5710 ts_tablet_manager.cc:1428] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260928 16:00:20.662686 5653 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:44197
I20260928 16:00:20.662786 5818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:44197 every 8 connection(s)
I20260928 16:00:20.663872 5653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:00:20.664983 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5653
I20260928 16:00:20.665786 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:41701
--local_ip_for_outbound_sockets=127.22.125.194
--tserver_master_addrs=127.22.125.254:39405
--webserver_port=40557
--webserver_interface=127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:20.669458 5821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:20.667066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:43107)
0928 16:00:20.667990 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:20.668304 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":336,"threads_started":1}
W20260928 16:00:20.670135 5819 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:39405 (0 consecutive failures): Network error: Failed to ping master at 127.22.125.254:39405: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
I20260928 16:00:20.671139 5821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:20.670720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:48191)
0928 16:00:20.670969 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:20.671039 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260928 16:00:20.672179 5821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:20.671758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:46115)
0928 16:00:20.671930 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:20.672011 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260928 16:00:20.672487 5819 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:00:20.986611 5822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:20.986865 5822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:20.986990 5822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:20.993978 5822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:20.994074 5822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:20.994279 5822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:00:21.003475 5822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:41701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=40557
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:21.004663 5822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:21.005975 5822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:21.018610 5830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:21.018697 5827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:21.018663 5828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:21.019599 5822 server_base.cc:1034] running on GCE node
I20260928 16:00:21.020247 5822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:21.021091 5822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:21.022290 5822 hybrid_clock.cc:648] HybridClock initialized: now 1790611221022250 us; error 29 us; skew 500 ppm
I20260928 16:00:21.022557 5822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:21 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611221, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:21.025561 5822 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:00:21.026996 5822 webserver.cc:460] Webserver started at http://127.22.125.194:40557/ using document root <none> and password file <none>
I20260928 16:00:21.027606 5822 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:21.027716 5822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:21.032919 5822 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:00:21.035493 5837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:21.036826 5822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:00:21.037001 5822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "f374783c869641afbd63a8d0027467b1"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "63cfac90035cd463dccac1d8495b8b98"
server_key_iv: "3d8aff717bc93b0e52e870bf68c43c74"
server_key_version: "encryptionkey@0"
I20260928 16:00:21.037964 5822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:21.051436 5822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:21.055760 5822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:21.056075 5822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:21.057022 5822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:21.058303 5822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:21.058390 5822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:21.058537 5822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:21.058619 5822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:21.092017 5822 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:41701
I20260928 16:00:21.092136 5949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:41701 every 8 connection(s)
I20260928 16:00:21.093528 5822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:00:21.098317 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:21.096528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:57759)
0928 16:00:21.097248 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:21.097461 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":268,"threads_started":1}
W20260928 16:00:21.098974 5950 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:39405 (0 consecutive failures): Network error: Failed to ping master at 127.22.125.254:39405: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
I20260928 16:00:21.099581 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5822
I20260928 16:00:21.099887 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:21.099518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:51969)
0928 16:00:21.099671 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:21.099786 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 16:00:21.100361 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:36647
--local_ip_for_outbound_sockets=127.22.125.195
--tserver_master_addrs=127.22.125.254:39405
--webserver_port=37237
--webserver_interface=127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:00:21.100752 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:21.100423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:37055)
0928 16:00:21.100562 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:21.100641 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260928 16:00:21.100996 5950 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:00:21.421363 5953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:21.421648 5953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:21.421744 5953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:21.428916 5953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:21.429013 5953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:21.429129 5953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:00:21.438679 5953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:36647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=37237
--tserver_master_addrs=127.22.125.254:39405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:21.439838 5953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:21.441120 5953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:21.454092 5961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:21.454069 5959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:21.454563 5958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:21.455008 5953 server_base.cc:1034] running on GCE node
I20260928 16:00:21.455576 5953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:21.456489 5953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:21.457679 5953 hybrid_clock.cc:648] HybridClock initialized: now 1790611221457616 us; error 60 us; skew 500 ppm
I20260928 16:00:21.457934 5953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:21 dist-test-slave-9cqd krb5kdc[4496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611221, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:21.461068 5953 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:00:21.462391 5953 webserver.cc:460] Webserver started at http://127.22.125.195:37237/ using document root <none> and password file <none>
I20260928 16:00:21.462996 5953 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:21.463093 5953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:21.468158 5953 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 16:00:21.470597 5968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:21.471866 5953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 16:00:21.472097 5953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "fe73d580f8ed4ea2bb4af9585b43d79e"
format_stamp: "Formatted at 2026-09-28 16:00:15 on dist-test-slave-9cqd"
server_key: "116dc073588944550895b5b7b414b2a3"
server_key_iv: "72fe64f92a9eaebe4359b751ccd1d71d"
server_key_version: "encryptionkey@0"
I20260928 16:00:21.473052 5953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:21.488569 5953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:21.492903 5953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:21.493182 5953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:21.494226 5953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:21.495505 5953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:21.495627 5953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:21.495766 5953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:21.495867 5953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:21.528201 5953 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:36647
I20260928 16:00:21.528303 6080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:36647 every 8 connection(s)
I20260928 16:00:21.529672 5953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:00:21.532222 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 5953
I20260928 16:00:21.534917 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:21.532834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:44451)
0928 16:00:21.533771 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:21.534073 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":371,"threads_started":1}
W20260928 16:00:21.535490 6081 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:39405 (0 consecutive failures): Network error: Failed to ping master at 127.22.125.254:39405: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
I20260928 16:00:21.536271 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:21.535923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:51145)
0928 16:00:21.536100 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:21.536176 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 16:00:21.537017 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:21.536723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:41119)
0928 16:00:21.536837 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:21.536913 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260928 16:00:21.537310 6081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 16:00:21.673635 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:21.672939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:44963)
0928 16:00:21.673346 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:21.673429 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":198,"threads_started":1}
I20260928 16:00:22.102361 6086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:22.101497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:36927)
0928 16:00:22.101923 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:22.102047 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":204,"threads_started":1}
I20260928 16:00:22.316610 6087 raft_consensus.cc:482] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:22.316919 6087 raft_consensus.cc:504] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:22.318434 6087 leader_election.cc:288] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647), f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701)
Sep 28 16:00:22 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
Sep 28 16:00:22 dist-test-slave-9cqd krb5kdc[4496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 16:00:22.348811 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:22.318625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:36647 (local address 127.22.125.193:59475)
0928 16:00:22.329082 (+ 10457us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:22.329139 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:22.329767 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:22.332386 (+ 2619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:22.332399 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:22.333389 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:22.334063 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:22.334082 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.335605 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.335608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:22.337276 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:22.337288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.338298 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.338307 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:22.338599 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:22.339447 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:22.339464 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:22.341917 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:22.345527 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:22.345537 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:22.345551 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:22.346062 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:22.346499 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:22.346504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:22.346508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:22.346695 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:22.347568 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:22.347580 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:22.347775 (+ 195us) client_negotiation.cc:769] Sending connection context
0928 16:00:22.348129 (+ 354us) client_negotiation.cc:240] Negotiation successful
0928 16:00:22.348518 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10328,"mutex_wait_us":122,"spinlock_wait_cycles":4736,"thread_start_us":10236,"threads_started":1}
I20260928 16:00:22.349339 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:22.319179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:53029 (local address 127.22.125.194:41701)
0928 16:00:22.329526 (+ 10347us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:22.329541 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:22.330051 (+ 510us) server_negotiation.cc:407] Connection header received
0928 16:00:22.331675 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:22.331682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:22.332111 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:22.332432 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:22.334272 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.335412 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.337418 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.338027 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.338226 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:22.342553 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:22.342560 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:22.342571 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:22.342612 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:22.345443 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:22.346218 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:22.346225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:22.346233 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:22.346400 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:22.346832 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:22.346837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:22.346840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:22.347310 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:22.347506 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:22.348585 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0928 16:00:22.348932 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10164,"thread_start_us":9998,"threads_started":1}
I20260928 16:00:22.349355 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:22.318961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:59475 (local address 127.22.125.195:36647)
0928 16:00:22.328808 (+ 9847us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:22.328832 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:22.329638 (+ 806us) server_negotiation.cc:407] Connection header received
0928 16:00:22.331452 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:22.331461 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:22.331897 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:22.332286 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:22.334265 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.335393 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.337415 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.338027 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.338242 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:22.342287 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:22.342297 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:22.342310 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:22.342347 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:22.345376 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:22.346221 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:22.346227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:22.346232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:22.346380 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:22.346827 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:22.346831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:22.346833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:22.347320 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:22.347498 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:22.348523 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0928 16:00:22.348925 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9638,"thread_start_us":9515,"threads_started":1}
I20260928 16:00:22.350173 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:22.318763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:41701 (local address 127.22.125.193:53029)
0928 16:00:22.329927 (+ 11164us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:22.329942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:22.330085 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:22.332387 (+ 2302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:22.332398 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:22.333393 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:22.334062 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:22.334082 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.335596 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.335603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:22.337284 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:22.337296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:22.338242 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:22.338248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:22.338594 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:22.339376 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:22.339402 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:22.342305 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:22.345602 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:22.345608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:22.345612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:22.346065 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:22.346519 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:22.346526 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:22.346529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:22.346712 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:22.347460 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:22.347468 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:22.347785 (+ 317us) client_negotiation.cc:769] Sending connection context
0928 16:00:22.348132 (+ 347us) client_negotiation.cc:240] Negotiation successful
0928 16:00:22.349791 (+ 1659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11029,"mutex_wait_us":1436,"spinlock_wait_cycles":65920,"thread_start_us":10853,"threads_started":1}
W20260928 16:00:22.352365 5707 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
W20260928 16:00:22.352948 5706 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:22.353109 5706 leader_election.cc:302] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e; no voters: f374783c869641afbd63a8d0027467b1, fe73d580f8ed4ea2bb4af9585b43d79e
I20260928 16:00:22.353462 6087 raft_consensus.cc:2726] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:00:22.538552 6092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:22.537767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:38657)
0928 16:00:22.538193 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:22.538322 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":198,"threads_started":1}
I20260928 16:00:22.674945 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:22.674485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:50659)
0928 16:00:22.674661 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:22.674762 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 16:00:23.104151 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:23.103382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:47471)
0928 16:00:23.103823 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:23.103925 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":239,"threads_started":1}
I20260928 16:00:23.540159 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:23.539501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:51357)
0928 16:00:23.539919 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:23.539993 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":226,"threads_started":1}
I20260928 16:00:23.676626 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:23.675866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:35037)
0928 16:00:23.676338 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:23.676415 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":277,"threads_started":1}
I20260928 16:00:24.105659 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:24.104969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:38271)
0928 16:00:24.105402 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:24.105479 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":261,"threads_started":1}
I20260928 16:00:24.541775 6097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:24.540919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:55339)
0928 16:00:24.541445 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:24.541523 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":263,"threads_started":1}
I20260928 16:00:24.547611 6098 raft_consensus.cc:482] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:24.547798 6098 raft_consensus.cc:504] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:24.548470 6098 leader_election.cc:288] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647), f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701)
W20260928 16:00:24.549304 5707 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
W20260928 16:00:24.549463 5706 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:24.549606 5706 leader_election.cc:302] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e; no voters: f374783c869641afbd63a8d0027467b1, fe73d580f8ed4ea2bb4af9585b43d79e
I20260928 16:00:24.549852 6098 raft_consensus.cc:2726] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:00:24.678151 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:24.677378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:43317)
0928 16:00:24.677876 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:24.677980 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":214,"threads_started":1}
I20260928 16:00:25.107272 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:25.106546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:33873)
0928 16:00:25.106995 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:25.107079 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260928 16:00:25.543258 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:25.542587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:38949)
0928 16:00:25.543010 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:25.543092 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":209,"threads_started":1}
I20260928 16:00:25.679811 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:25.678978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:41369)
0928 16:00:25.679542 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:25.679637 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":309,"threads_started":1}
I20260928 16:00:26.109282 6103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:26.108363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:45621)
0928 16:00:26.109047 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:26.109122 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":495,"threads_started":1}
I20260928 16:00:26.544785 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:26.544130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:47041)
0928 16:00:26.544548 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:26.544621 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":222,"threads_started":1}
I20260928 16:00:26.681286 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:26.680612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:58901)
0928 16:00:26.681046 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:26.681125 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":237,"threads_started":1}
I20260928 16:00:27.110920 6106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:27.110170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:51899)
0928 16:00:27.110636 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:27.110735 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
I20260928 16:00:27.161480 6107 raft_consensus.cc:482] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:27.161715 6107 raft_consensus.cc:504] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:27.162420 6107 leader_election.cc:288] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647), f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701)
W20260928 16:00:27.163307 5706 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
W20260928 16:00:27.163515 5707 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:27.163618 5707 leader_election.cc:302] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e; no voters: f374783c869641afbd63a8d0027467b1, fe73d580f8ed4ea2bb4af9585b43d79e
I20260928 16:00:27.163834 6107 raft_consensus.cc:2726] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:00:27.546370 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:27.545661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:43795)
0928 16:00:27.546121 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:27.546196 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":248,"threads_started":1}
I20260928 16:00:27.682787 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:27.682127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:43493)
0928 16:00:27.682513 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:27.682604 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":199,"threads_started":1}
I20260928 16:00:28.112850 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:28.111900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:50549)
0928 16:00:28.112551 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:28.112644 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":366,"threads_started":1}
I20260928 16:00:28.548032 6111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:28.547256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:56835)
0928 16:00:28.547773 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:28.547860 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":244,"threads_started":1}
I20260928 16:00:28.684526 6112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:28.683568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:39449)
0928 16:00:28.684251 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:28.684349 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":470,"threads_started":1}
I20260928 16:00:29.114506 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:29.113800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:47127)
0928 16:00:29.114186 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:29.114274 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":197,"threads_started":1}
I20260928 16:00:29.549597 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:29.548914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:35899)
0928 16:00:29.549333 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:29.549415 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":216,"threads_started":1}
I20260928 16:00:29.643656 6115 raft_consensus.cc:482] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:29.643864 6115 raft_consensus.cc:504] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:29.644520 6115 leader_election.cc:288] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647), f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701)
W20260928 16:00:29.645398 5707 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
W20260928 16:00:29.645557 5706 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:29.645718 5706 leader_election.cc:302] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e; no voters: f374783c869641afbd63a8d0027467b1, fe73d580f8ed4ea2bb4af9585b43d79e
I20260928 16:00:29.645961 6115 raft_consensus.cc:2726] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:00:29.685989 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:29.685257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:57857)
0928 16:00:29.685704 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:29.685801 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":228,"threads_started":1}
I20260928 16:00:30.116350 6117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:30.115459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:38155)
0928 16:00:30.116084 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:30.116194 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":318,"threads_started":1}
I20260928 16:00:30.551199 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:30.550383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:33949)
0928 16:00:30.550971 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:30.551049 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":338,"threads_started":1}
I20260928 16:00:30.687633 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:30.686853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:46579)
0928 16:00:30.687363 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:30.687442 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":297,"threads_started":1}
I20260928 16:00:31.117928 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:31.117252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:38387)
0928 16:00:31.117667 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:31.117746 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":229,"threads_started":1}
I20260928 16:00:31.552873 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:31.552200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:45447)
0928 16:00:31.552611 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:31.552697 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":212,"threads_started":1}
I20260928 16:00:31.689278 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:31.688486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:52589)
0928 16:00:31.688978 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:31.689096 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":225,"threads_started":1}
I20260928 16:00:32.119542 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:32.118797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:55297)
0928 16:00:32.119299 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:32.119388 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":276,"threads_started":1}
I20260928 16:00:32.554340 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:32.553641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:47633)
0928 16:00:32.554105 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:32.554187 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":243,"threads_started":1}
I20260928 16:00:32.690982 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:32.690131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:52843)
0928 16:00:32.690659 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:32.690770 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":299,"threads_started":1}
I20260928 16:00:33.121236 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:33.120509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:46413)
0928 16:00:33.120965 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:33.121057 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":228,"threads_started":1}
I20260928 16:00:33.555964 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:33.555254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:57337)
0928 16:00:33.555700 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:33.555787 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":242,"threads_started":1}
I20260928 16:00:33.692574 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:33.691735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:41245)
0928 16:00:33.692253 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:33.692369 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":237,"threads_started":1}
I20260928 16:00:34.122956 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:34.122276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:50143)
0928 16:00:34.122713 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:34.122799 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":229,"threads_started":1}
I20260928 16:00:34.557490 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:34.556807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:50443)
0928 16:00:34.557248 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:34.557334 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":219,"threads_started":1}
I20260928 16:00:34.694271 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:34.693411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:50017)
0928 16:00:34.693901 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:34.694009 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":259,"threads_started":1}
I20260928 16:00:34.766191 6132 raft_consensus.cc:482] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:34.766451 6132 raft_consensus.cc:504] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73d580f8ed4ea2bb4af9585b43d79e" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36647 } } peers { permanent_uuid: "f374783c869641afbd63a8d0027467b1" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 41701 } } peers { permanent_uuid: "38a11e64b7214e35b5dcab8d8ff9a03e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 44197 } }
I20260928 16:00:34.767117 6132 leader_election.cc:288] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647), f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701)
W20260928 16:00:34.768095 5707 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe73d580f8ed4ea2bb4af9585b43d79e (127.22.125.195:36647): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
W20260928 16:00:34.768260 5706 leader_election.cc:341] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f374783c869641afbd63a8d0027467b1 (127.22.125.194:41701): Not found: Tablet not found: 9287a971b89c4e8e976d6e96aa073d8c
I20260928 16:00:34.768415 5706 leader_election.cc:302] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38a11e64b7214e35b5dcab8d8ff9a03e; no voters: f374783c869641afbd63a8d0027467b1, fe73d580f8ed4ea2bb4af9585b43d79e
I20260928 16:00:34.768674 6132 raft_consensus.cc:2726] T 9287a971b89c4e8e976d6e96aa073d8c P 38a11e64b7214e35b5dcab8d8ff9a03e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:00:35.124750 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:35.124005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:36983)
0928 16:00:35.124486 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:35.124563 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":221,"threads_started":1}
I20260928 16:00:35.559067 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:35.558389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:54021)
0928 16:00:35.558802 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:35.558876 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":203,"threads_started":1}
I20260928 16:00:35.696152 6135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:35.695411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.193:56233)
0928 16:00:35.695836 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:35.695916 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
I20260928 16:00:36.126482 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:36.125733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.194:43675)
0928 16:00:36.126187 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:36.126308 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":204,"threads_started":1}
I20260928 16:00:36.541759 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 5653
I20260928 16:00:36.563122 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:36.544370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50088 (local address 127.22.125.193:44197)
0928 16:00:36.544981 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:36.544993 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:36.545027 (+ 34us) server_negotiation.cc:407] Connection header received
0928 16:00:36.545109 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:36.545121 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:36.545442 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:36.545639 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:36.547059 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:36.548278 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:36.549975 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:36.551059 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:36.551186 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:36.554781 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:36.554795 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:36.554813 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:36.554860 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:36.558894 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:36.559733 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:36.559743 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:36.559749 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:36.559953 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:36.560506 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:36.560513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:36.560518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:36.561009 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:36.561333 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:36.562586 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0928 16:00:36.562863 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":703,"threads_started":1}
I20260928 16:00:36.564692 5814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:36.564821 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:36.560203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:39405 (local address 127.22.125.195:34271)
0928 16:00:36.564490 (+ 4287us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:36.564635 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:39405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4143,"threads_started":1}
I20260928 16:00:36.653910 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 5653
I20260928 16:00:36.671401 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 5822
I20260928 16:00:36.684475 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:36.672179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35356 (local address 127.22.125.194:41701)
0928 16:00:36.672692 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:36.672697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:36.672713 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:00:36.672767 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:36.672771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:36.672832 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:36.672925 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:36.674191 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:36.674983 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:36.676263 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:36.676718 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:36.676815 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:36.679411 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:36.679416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:36.679420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:36.679453 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:36.682020 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:36.682687 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:36.682691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:36.682694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:36.682813 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:36.683174 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:36.683177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:36.683179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:36.683430 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:36.683628 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:36.683860 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 16:00:36.684015 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":300,"threads_started":1}
I20260928 16:00:36.685200 5945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:36.769470 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 5822
I20260928 16:00:36.787011 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 5953
I20260928 16:00:36.800271 6142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:36.787897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.22.125.195:36647)
0928 16:00:36.788526 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:36.788531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:36.788549 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:00:36.788607 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:36.788610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:36.788669 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:36.788763 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:36.790169 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:36.790950 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:36.792300 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:36.792727 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:36.792832 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:36.795457 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:36.795461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:36.795464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:36.795491 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:36.798007 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:36.798650 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:36.798653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:36.798655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:36.798770 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:36.799100 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:36.799103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:36.799105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:36.799353 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:36.799550 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:36.799755 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 16:00:36.799925 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":357,"threads_started":1}
I20260928 16:00:36.800958 6076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:36.884250 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 5953
2026-09-28T16:00:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24241 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:00:36 dist-test-slave-9cqd krb5kdc[6146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:00:36 dist-test-slave-9cqd krb5kdc[6146](info): set up 2 sockets
Sep 28 16:00:36 dist-test-slave-9cqd krb5kdc[6146](info): commencing operation
krb5kdc: starting...
W20260928 16:00:37.986557 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:00:38 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:00:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:00:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:00:38.227934 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45811
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:45811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:38.559866 6159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:38.560175 6159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:38.560335 6159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:38.567503 6159 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 16:00:38.567592 6159 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 16:00:38.567683 6159 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 16:00:38.567763 6159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:00:38.567821 6159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:38.567888 6159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:38.567934 6159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:00:38.567986 6159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:00:38.577503 6159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:45811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:38.578729 6159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:38.580000 6159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:38.592063 6167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:38.592125 6165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:38.592159 6164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:38.592927 6159 server_base.cc:1034] running on GCE node
I20260928 16:00:38.593724 6159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:38.595099 6159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:38.596316 6159 hybrid_clock.cc:648] HybridClock initialized: now 1790611238596259 us; error 35 us; skew 500 ppm
I20260928 16:00:38.596665 6159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:38 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611238, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:38.606480 6159 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:00:38.608028 6159 webserver.cc:460] Webserver started at http://127.22.125.254:36361/ using document root <none> and password file <none>
I20260928 16:00:38.608788 6159 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:38.608920 6159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:38.609292 6159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:38.612355 6159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "91f34e2a31b44ef89b5297d57c752433"
format_stamp: "Formatted at 2026-09-28 16:00:38 on dist-test-slave-9cqd"
server_key: "2eb02a56f29595e023813d3406dd9ce3"
server_key_iv: "9408bb3d9802b1f33aed9e3db3a44a55"
server_key_version: "encryptionkey@0"
I20260928 16:00:38.613155 6159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "91f34e2a31b44ef89b5297d57c752433"
format_stamp: "Formatted at 2026-09-28 16:00:38 on dist-test-slave-9cqd"
server_key: "2eb02a56f29595e023813d3406dd9ce3"
server_key_iv: "9408bb3d9802b1f33aed9e3db3a44a55"
server_key_version: "encryptionkey@0"
I20260928 16:00:38.618196 6159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:00:38.621572 6174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:38.622800 6159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:00:38.622982 6159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "91f34e2a31b44ef89b5297d57c752433"
format_stamp: "Formatted at 2026-09-28 16:00:38 on dist-test-slave-9cqd"
server_key: "2eb02a56f29595e023813d3406dd9ce3"
server_key_iv: "9408bb3d9802b1f33aed9e3db3a44a55"
server_key_version: "encryptionkey@0"
I20260928 16:00:38.623198 6159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:38.645185 6159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:38.649019 6159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:38.649293 6159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:38.668203 6159 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:45811
I20260928 16:00:38.668226 6225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:45811 every 8 connection(s)
I20260928 16:00:38.669497 6159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:00:38.673178 6226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:38.679682 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 6159
I20260928 16:00:38.679912 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:00:38.680630 23031 external_mini_cluster.cc:1428] Setting key 049a007cd8bfbfca09ab171e2cf7b6c9
I20260928 16:00:38.683889 6226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Bootstrap starting.
I20260928 16:00:38.687892 6226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:38.689311 6226 log.cc:824] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:38.692152 6226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: No bootstrap required, opened a new log
Sep 28 16:00:38 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:38.701016 6226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:00:38.701493 6226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:38.701604 6226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f34e2a31b44ef89b5297d57c752433, State: Initialized, Role: FOLLOWER
I20260928 16:00:38.702282 6226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:00:38.702597 6226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:00:38.702771 6226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:00:38.702953 6226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:38.705437 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:38.682713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.22.125.254:45811)
0928 16:00:38.683555 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:38.683570 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:38.683621 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:00:38.686130 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:38.686159 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:38.686694 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:38.687110 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:38.688563 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:38.690430 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:38.691838 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:38.692910 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:38.693098 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:38.697588 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:38.697596 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:38.697610 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:38.697652 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:38.700849 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:38.701725 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:38.701734 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:38.701742 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:38.701975 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:38.702441 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:38.702444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:38.702446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:38.703186 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:38.703400 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:38.703901 (+ 501us) server_negotiation.cc:299] Negotiation successful
0928 16:00:38.704399 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":355,"threads_started":1}
I20260928 16:00:38.705721 6226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:00:38.706246 6226 leader_election.cc:302] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91f34e2a31b44ef89b5297d57c752433; no voters:
I20260928 16:00:38.706892 6226 leader_election.cc:288] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:00:38.707026 6231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:00:38.709132 6226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:00:38.712733 6231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 LEADER]: Becoming Leader. State: Replica: 91f34e2a31b44ef89b5297d57c752433, State: Running, Role: LEADER
I20260928 16:00:38.713352 6231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:00:38.717388 6233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91f34e2a31b44ef89b5297d57c752433. Latest consensus state: current_term: 1 leader_uuid: "91f34e2a31b44ef89b5297d57c752433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } } }
I20260928 16:00:38.717772 6233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: This master's current role is: LEADER
I20260928 16:00:38.718305 6232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91f34e2a31b44ef89b5297d57c752433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } } }
I20260928 16:00:38.718614 6232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: This master's current role is: LEADER
W20260928 16:00:38.722177 6245 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:00:38.722278 6245 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:00:38.722478 6246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:00:38.724207 6246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:00:38.732013 6246 catalog_manager.cc:1347] Generated new cluster ID: 949761a23cc24b3ca723e920706c03d8
I20260928 16:00:38.732146 6246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:00:38.751516 6246 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:00:38.753461 6246 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:00:38.768071 6246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Generated new TSK 0
I20260928 16:00:38.768936 6246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 16:00:38.843186 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45811
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:39.189424 6252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:39.189836 6252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:39.189956 6252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:39.197155 6252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:39.197265 6252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:39.197386 6252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:00:39.207541 6252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:39.208995 6252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:39.210498 6252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:39.224066 6257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:39.224014 6260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:39.224133 6258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:39.225081 6252 server_base.cc:1034] running on GCE node
I20260928 16:00:39.225631 6252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:39.226577 6252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:39.227857 6252 hybrid_clock.cc:648] HybridClock initialized: now 1790611239227801 us; error 39 us; skew 500 ppm
I20260928 16:00:39.228242 6252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:39 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611239, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:39.231366 6252 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:00:39.232843 6252 webserver.cc:460] Webserver started at http://127.22.125.193:37853/ using document root <none> and password file <none>
I20260928 16:00:39.233520 6252 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:39.233678 6252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:39.233981 6252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:39.236907 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "26a46693e0f9430b85f4df38e8375138"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "4a5fa37c917ec736af6f6212edc0672e"
server_key_iv: "24f930e0d7f058a4a66ff43862f726b6"
server_key_version: "encryptionkey@0"
I20260928 16:00:39.237670 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "26a46693e0f9430b85f4df38e8375138"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "4a5fa37c917ec736af6f6212edc0672e"
server_key_iv: "24f930e0d7f058a4a66ff43862f726b6"
server_key_version: "encryptionkey@0"
I20260928 16:00:39.242772 6252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:00:39.246188 6267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:39.247488 6252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:00:39.247677 6252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "26a46693e0f9430b85f4df38e8375138"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "4a5fa37c917ec736af6f6212edc0672e"
server_key_iv: "24f930e0d7f058a4a66ff43862f726b6"
server_key_version: "encryptionkey@0"
I20260928 16:00:39.247877 6252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:39.294658 6252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:39.298677 6252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:39.298980 6252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:39.300064 6252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:39.301395 6252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:39.301508 6252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:39.301668 6252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:39.301744 6252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:39.337209 6252 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:38495
I20260928 16:00:39.337242 6379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:38495 every 8 connection(s)
I20260928 16:00:39.338541 6252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:00:39.340503 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 6252
I20260928 16:00:39.340732 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:00:39.341549 23031 external_mini_cluster.cc:1428] Setting key 60758956bb54ed1c85454838c7ea4d04
Sep 28 16:00:39 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611239, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:39.359279 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:39.341253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:59613 (local address 127.22.125.254:45811)
0928 16:00:39.341779 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:39.341787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:39.342846 (+ 1059us) server_negotiation.cc:407] Connection header received
0928 16:00:39.344812 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:39.344818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:39.344921 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:39.345043 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:39.346644 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.347383 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.349041 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.349439 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.349583 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:39.353527 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:39.353532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:39.353535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:39.353568 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:39.356225 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:39.356955 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:39.356961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:39.356964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:39.357091 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:39.357539 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:39.357548 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:39.357551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:39.357806 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:39.357975 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:39.358670 (+ 695us) server_negotiation.cc:299] Negotiation successful
0928 16:00:39.358906 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":241,"threads_started":1}
I20260928 16:00:39.360332 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:39.341865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:59613)
0928 16:00:39.342671 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:39.342724 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:39.344536 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:39.345313 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:39.345323 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:39.345867 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:39.346464 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:39.346482 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.347563 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.347568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:39.348883 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:39.348896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.349674 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.349681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:39.349916 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:39.350608 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:39.350641 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:39.353303 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:39.356363 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:39.356374 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:39.356390 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:39.356808 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:39.357215 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:39.357220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:39.357224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:39.357411 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:39.357945 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:39.357952 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:39.358294 (+ 342us) client_negotiation.cc:769] Sending connection context
0928 16:00:39.358564 (+ 270us) client_negotiation.cc:240] Negotiation successful
0928 16:00:39.359104 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":343,"threads_started":1}
I20260928 16:00:39.362012 6380 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45811
I20260928 16:00:39.362317 6380 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:39.363035 6380 heartbeater.cc:511] Master 127.22.125.254:45811 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 16:00:39.365335 6191 ts_manager.cc:194] Registered new tserver with Master: 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495)
I20260928 16:00:39.366933 6191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:59613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 16:00:39.394570 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45811
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:39.715447 6386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:39.715703 6386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:39.715795 6386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:39.722934 6386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:39.723029 6386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:39.723137 6386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:00:39.732719 6386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:39.733873 6386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:39.735098 6386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:39.748334 6391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:39.748297 6392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:39.748260 6394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:39.749531 6386 server_base.cc:1034] running on GCE node
I20260928 16:00:39.750106 6386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:39.751072 6386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:39.752293 6386 hybrid_clock.cc:648] HybridClock initialized: now 1790611239752253 us; error 35 us; skew 500 ppm
I20260928 16:00:39.752611 6386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:39 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611239, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:39.755664 6386 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:00:39.757107 6386 webserver.cc:460] Webserver started at http://127.22.125.194:35113/ using document root <none> and password file <none>
I20260928 16:00:39.757767 6386 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:39.757920 6386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:39.758219 6386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:39.761126 6386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "76093d4c30d44531bfaa251d001c881f"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "94f205a2e968bc1374dba28e98b67c5d"
server_key_iv: "52a1b6814657fda5b69acf17251d0e97"
server_key_version: "encryptionkey@0"
I20260928 16:00:39.761868 6386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "76093d4c30d44531bfaa251d001c881f"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "94f205a2e968bc1374dba28e98b67c5d"
server_key_iv: "52a1b6814657fda5b69acf17251d0e97"
server_key_version: "encryptionkey@0"
I20260928 16:00:39.766894 6386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:00:39.770233 6401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:39.771489 6386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260928 16:00:39.771660 6386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "76093d4c30d44531bfaa251d001c881f"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "94f205a2e968bc1374dba28e98b67c5d"
server_key_iv: "52a1b6814657fda5b69acf17251d0e97"
server_key_version: "encryptionkey@0"
I20260928 16:00:39.771857 6386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:39.785924 6386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:39.790293 6386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:39.790611 6386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:39.791643 6386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:39.793004 6386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:39.793157 6386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:39.793313 6386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:39.793407 6386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:39.826186 6386 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:39101
I20260928 16:00:39.826403 6514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:39101 every 8 connection(s)
I20260928 16:00:39.827661 6386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:00:39.837488 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 6386
I20260928 16:00:39.837641 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:00:39.838030 23031 external_mini_cluster.cc:1428] Setting key bed82f88c34296395ef188a4b29c5677
Sep 28 16:00:39 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611239, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:39.848031 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:39.830300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:57579 (local address 127.22.125.254:45811)
0928 16:00:39.830524 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:39.830531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:39.831678 (+ 1147us) server_negotiation.cc:407] Connection header received
0928 16:00:39.833449 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:39.833453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:39.833540 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:39.833626 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:39.835191 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.835907 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.837451 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.837999 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.838146 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:39.842016 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:39.842021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:39.842025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:39.842055 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:39.844719 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:39.845667 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:39.845672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:39.845695 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:39.845818 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:39.846287 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:39.846296 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:39.846299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:39.846562 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:39.846747 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:39.847602 (+ 855us) server_negotiation.cc:299] Negotiation successful
0928 16:00:39.847779 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 16:00:39.849627 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:39.830812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:57579)
0928 16:00:39.831521 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:39.831557 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:39.833187 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:39.833859 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:39.833867 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:39.834373 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:39.834919 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:39.834936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.836069 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.836073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:39.837293 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:39.837305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:39.838228 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:39.838236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:39.838487 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:39.839323 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:39.839351 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:39.841795 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:39.844906 (+ 3111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:39.845017 (+ 111us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:39.845030 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:39.845512 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:39.845940 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:39.845945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:39.845949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:39.846153 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:39.846696 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:39.846704 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:39.847162 (+ 458us) client_negotiation.cc:769] Sending connection context
0928 16:00:39.847488 (+ 326us) client_negotiation.cc:240] Negotiation successful
0928 16:00:39.848101 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":262,"threads_started":1}
I20260928 16:00:39.851271 6515 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45811
I20260928 16:00:39.851604 6515 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:39.852555 6515 heartbeater.cc:511] Master 127.22.125.254:45811 requested a full tablet report, sending...
I20260928 16:00:39.854115 6191 ts_manager.cc:194] Registered new tserver with Master: 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101)
I20260928 16:00:39.854975 6191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:57579
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 16:00:39.891605 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45811
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:00:40.206563 6520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:00:40.206902 6520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:00:40.207064 6520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:00:40.214745 6520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:00:40.214890 6520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:00:40.215060 6520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:00:40.224737 6520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:00:40.225978 6520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:00:40.227407 6520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:00:40.240124 6525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:40.240141 6528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:00:40.240201 6526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:00:40.240773 6520 server_base.cc:1034] running on GCE node
I20260928 16:00:40.241415 6520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:00:40.242292 6520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:00:40.243552 6520 hybrid_clock.cc:648] HybridClock initialized: now 1790611240243493 us; error 45 us; skew 500 ppm
I20260928 16:00:40.243893 6520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:00:40 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611240, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:00:40.246958 6520 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:00:40.248294 6520 webserver.cc:460] Webserver started at http://127.22.125.195:40577/ using document root <none> and password file <none>
I20260928 16:00:40.248894 6520 fs_manager.cc:359] Metadata directory not provided
I20260928 16:00:40.248991 6520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:00:40.249230 6520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:00:40.252090 6520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "9affaea8f0094261be224a439969eca0"
format_stamp: "Formatted at 2026-09-28 16:00:40 on dist-test-slave-9cqd"
server_key: "9b9fd5f5ed5cefb7950e7907c0aad765"
server_key_iv: "111326e18386bcd8ad90c8b1d4eb19b1"
server_key_version: "encryptionkey@0"
I20260928 16:00:40.252923 6520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "9affaea8f0094261be224a439969eca0"
format_stamp: "Formatted at 2026-09-28 16:00:40 on dist-test-slave-9cqd"
server_key: "9b9fd5f5ed5cefb7950e7907c0aad765"
server_key_iv: "111326e18386bcd8ad90c8b1d4eb19b1"
server_key_version: "encryptionkey@0"
I20260928 16:00:40.257663 6520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260928 16:00:40.260931 6535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:40.262142 6520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:00:40.262318 6520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "9affaea8f0094261be224a439969eca0"
format_stamp: "Formatted at 2026-09-28 16:00:40 on dist-test-slave-9cqd"
server_key: "9b9fd5f5ed5cefb7950e7907c0aad765"
server_key_iv: "111326e18386bcd8ad90c8b1d4eb19b1"
server_key_version: "encryptionkey@0"
I20260928 16:00:40.262518 6520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:00:40.286756 6520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:00:40.290640 6520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:00:40.290949 6520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:00:40.291992 6520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:00:40.293306 6520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:00:40.293418 6520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:40.293565 6520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:00:40.293640 6520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:00:40.326325 6520 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:46851
I20260928 16:00:40.326459 6647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:46851 every 8 connection(s)
I20260928 16:00:40.327692 6520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:00:40.332791 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 6520
I20260928 16:00:40.332988 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:00:40.333475 23031 external_mini_cluster.cc:1428] Setting key b1b5ffdfc776c59dbf24532dea80fd4f
Sep 28 16:00:40 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611240, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:00:40.351986 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.330423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:35689 (local address 127.22.125.254:45811)
0928 16:00:40.330605 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.330610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.331757 (+ 1147us) server_negotiation.cc:407] Connection header received
0928 16:00:40.333748 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.333753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.333855 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.333977 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:40.335636 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.336870 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.342224 (+ 5354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.342640 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.342746 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.346128 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:40.346133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:40.346136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:40.346169 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:40.349083 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.349881 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.349886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.349888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.350017 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.350443 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.350448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.350452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.350722 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:40.350838 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.351575 (+ 737us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.351739 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:00:40.352965 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.330924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:35689)
0928 16:00:40.331603 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:40.331638 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:40.333444 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:40.334205 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:40.334212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:40.334708 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:00:40.335434 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.335462 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.340185 (+ 4723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.340190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.342062 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.342077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.342844 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.342849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.343043 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.343671 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:00:40.343692 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:00:40.345918 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:00:40.349247 (+ 3329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:40.349254 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:40.349289 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:40.349759 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:40.350111 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:00:40.350114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:00:40.350116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:00:40.350316 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:00:40.350832 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:00:40.350838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:00:40.351150 (+ 312us) client_negotiation.cc:769] Sending connection context
0928 16:00:40.351425 (+ 275us) client_negotiation.cc:240] Negotiation successful
0928 16:00:40.351812 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":262,"threads_started":1}
I20260928 16:00:40.354331 6648 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45811
I20260928 16:00:40.354620 6648 heartbeater.cc:464] Registering TS with master...
I20260928 16:00:40.355288 6648 heartbeater.cc:511] Master 127.22.125.254:45811 requested a full tablet report, sending...
I20260928 16:00:40.356667 6191 ts_manager.cc:194] Registered new tserver with Master: 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851)
I20260928 16:00:40.357501 6191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:35689
I20260928 16:00:40.361632 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:00:40.369923 6380 heartbeater.cc:503] Master 127.22.125.254:45811 was elected leader, sending a full tablet report...
I20260928 16:00:40.378499 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.366048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.22.125.254:45811)
0928 16:00:40.366241 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.366249 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.366626 (+ 377us) server_negotiation.cc:407] Connection header received
0928 16:00:40.366832 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.366841 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.366900 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.367003 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:40.368482 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.369184 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.370718 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.371144 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.371288 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.373693 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:40.373698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:40.373701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:40.373730 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:40.376248 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.376920 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.376924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.376926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.377051 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.377381 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.377385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.377386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.377656 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:40.377833 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.378108 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.378265 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:00:40.382059 6191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:00:40.388936 6191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:00:40.412839 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.400583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.22.125.194:39101)
0928 16:00:40.402821 (+ 2238us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.402830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.402869 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:00:40.402962 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.402970 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.403255 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.404317 (+ 1062us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:40.404573 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.406614 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.409791 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.411276 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.411438 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.411611 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.412353 (+ 742us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.412521 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2070,"thread_start_us":1973,"threads_started":1}
I20260928 16:00:40.413056 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.400474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:39101 (local address 127.0.0.1:55072)
0928 16:00:40.401398 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:40.401445 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:40.401609 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:40.403522 (+ 1913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:40.403530 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:40.403570 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:40.404066 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.404086 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.406793 (+ 2707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.406802 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.409614 (+ 2812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.409629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.411772 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.411779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.412086 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.412164 (+ 78us) client_negotiation.cc:769] Sending connection context
0928 16:00:40.412303 (+ 139us) client_negotiation.cc:240] Negotiation successful
0928 16:00:40.412528 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":28,"thread_start_us":1027,"threads_started":1}
I20260928 16:00:40.415335 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.402278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:50400)
0928 16:00:40.407263 (+ 4985us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:40.407282 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:40.407410 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:40.408249 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:40.408253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:40.408277 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:40.408719 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.408731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.410552 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.410560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.412660 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.412678 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.414561 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.414567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.414732 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.414763 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 16:00:40.414873 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 16:00:40.414994 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4851,"thread_start_us":4720,"threads_started":1}
I20260928 16:00:40.415459 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.402877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50400 (local address 127.22.125.193:38495)
0928 16:00:40.403633 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.403644 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.407532 (+ 3888us) server_negotiation.cc:407] Connection header received
0928 16:00:40.407633 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.407640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.407891 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.408097 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:40.408915 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.410370 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.412865 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.414314 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.414526 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.414694 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.414928 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.415092 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":339,"threads_started":1}
I20260928 16:00:40.416649 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.401059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:46851 (local address 127.0.0.1:45544)
0928 16:00:40.402217 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:40.402233 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:40.402333 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:40.409313 (+ 6980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:40.409317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:40.409351 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:40.409825 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.409838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.411775 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.411779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.413929 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.413939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.415881 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.415888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.416081 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.416112 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 16:00:40.416179 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 16:00:40.416295 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":4,"thread_start_us":1087,"threads_started":1}
I20260928 16:00:40.417629 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.401835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.22.125.195:46851)
0928 16:00:40.408785 (+ 6950us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.408792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.408824 (+ 32us) server_negotiation.cc:407] Connection header received
0928 16:00:40.408932 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.408937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.409145 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.409323 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:40.410038 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.411611 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.414081 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.415527 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.416933 (+ 1406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.417087 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.417152 (+ 65us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.417303 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6775,"thread_start_us":6606,"threads_started":1}
I20260928 16:00:40.417318 6450 tablet_service.cc:1467] Processing CreateTablet for tablet a5431ea3664e4da187b6be66a3cfb395 (DEFAULT_TABLE table=test-table [id=52be0074fac1499ba69821c5302b7594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:40.418582 6450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5431ea3664e4da187b6be66a3cfb395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:40.418583 6315 tablet_service.cc:1467] Processing CreateTablet for tablet a5431ea3664e4da187b6be66a3cfb395 (DEFAULT_TABLE table=test-table [id=52be0074fac1499ba69821c5302b7594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:40.419587 6315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5431ea3664e4da187b6be66a3cfb395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:40.423626 6583 tablet_service.cc:1467] Processing CreateTablet for tablet a5431ea3664e4da187b6be66a3cfb395 (DEFAULT_TABLE table=test-table [id=52be0074fac1499ba69821c5302b7594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:00:40.425038 6583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5431ea3664e4da187b6be66a3cfb395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:00:40.431032 6666 tablet_bootstrap.cc:492] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Bootstrap starting.
I20260928 16:00:40.432935 6667 tablet_bootstrap.cc:492] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Bootstrap starting.
I20260928 16:00:40.434705 6666 tablet_bootstrap.cc:654] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:40.435969 6666 log.cc:824] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:40.436267 6667 tablet_bootstrap.cc:654] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:40.437374 6667 log.cc:824] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:40.439357 6666 tablet_bootstrap.cc:492] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: No bootstrap required, opened a new log
I20260928 16:00:40.439706 6666 ts_tablet_manager.cc:1397] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 16:00:40.440227 6669 tablet_bootstrap.cc:492] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Bootstrap starting.
I20260928 16:00:40.443146 6669 tablet_bootstrap.cc:654] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Neither blocks nor log segments found. Creating new log.
I20260928 16:00:40.443547 6667 tablet_bootstrap.cc:492] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: No bootstrap required, opened a new log
I20260928 16:00:40.443815 6667 ts_tablet_manager.cc:1397] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260928 16:00:40.444267 6669 log.cc:824] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Log is configured to *not* fsync() on all Append() calls
I20260928 16:00:40.446578 6669 tablet_bootstrap.cc:492] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: No bootstrap required, opened a new log
I20260928 16:00:40.446821 6669 ts_tablet_manager.cc:1397] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260928 16:00:40.450284 6666 raft_consensus.cc:348] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.450734 6666 raft_consensus.cc:374] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:40.450830 6666 raft_consensus.cc:729] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a46693e0f9430b85f4df38e8375138, State: Initialized, Role: FOLLOWER
I20260928 16:00:40.451436 6666 consensus_queue.cc:260] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.454121 6667 raft_consensus.cc:348] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.454499 6666 ts_tablet_manager.cc:1428] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Time spent starting tablet: real 0.015s user 0.004s sys 0.009s
I20260928 16:00:40.454609 6667 raft_consensus.cc:374] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:40.454748 6667 raft_consensus.cc:729] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76093d4c30d44531bfaa251d001c881f, State: Initialized, Role: FOLLOWER
I20260928 16:00:40.455075 6669 raft_consensus.cc:348] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.455494 6669 raft_consensus.cc:374] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:00:40.455386 6667 consensus_queue.cc:260] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.455596 6669 raft_consensus.cc:729] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9affaea8f0094261be224a439969eca0, State: Initialized, Role: FOLLOWER
I20260928 16:00:40.456183 6669 consensus_queue.cc:260] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.456619 6515 heartbeater.cc:503] Master 127.22.125.254:45811 was elected leader, sending a full tablet report...
I20260928 16:00:40.457827 6648 heartbeater.cc:503] Master 127.22.125.254:45811 was elected leader, sending a full tablet report...
I20260928 16:00:40.459559 6667 ts_tablet_manager.cc:1428] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260928 16:00:40.459712 6669 ts_tablet_manager.cc:1428] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
W20260928 16:00:40.579484 6649 tablet.cc:2369] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:00:40.579764 6516 tablet.cc:2369] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:00:40.590871 6381 tablet.cc:2369] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:00:40.604257 6673 raft_consensus.cc:482] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:00:40.604528 6673 raft_consensus.cc:504] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.606177 6673 leader_election.cc:288] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495), 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851)
I20260928 16:00:40.613408 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.606795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:60995 (local address 127.22.125.195:46851)
0928 16:00:40.607039 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.607045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.607780 (+ 735us) server_negotiation.cc:407] Connection header received
0928 16:00:40.607955 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.607960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.608064 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.608167 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:40.608778 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.609518 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.611442 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.612329 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.612451 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.612561 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.612971 (+ 410us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.613112 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 16:00:40.613456 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.606521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:38371 (local address 127.22.125.193:38495)
0928 16:00:40.606737 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.606745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.607551 (+ 806us) server_negotiation.cc:407] Connection header received
0928 16:00:40.607715 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.607723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.607807 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.607899 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:00:40.608561 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.609481 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.611563 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.612405 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.612570 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.612644 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.613083 (+ 439us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.613205 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:00:40.613572 6676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.606658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:46851 (local address 127.22.125.194:60995)
0928 16:00:40.607656 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:40.607679 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:40.607803 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:40.608193 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:40.608196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:40.608216 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:40.608617 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.608632 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.609715 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.609719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.611283 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.611295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.612549 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.612556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.612782 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.612806 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 16:00:40.612927 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 16:00:40.613123 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":1,"thread_start_us":559,"threads_started":1}
I20260928 16:00:40.613618 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.606548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.22.125.194:38371)
0928 16:00:40.607408 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0928 16:00:40.607431 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:00:40.607570 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:00:40.607939 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:00:40.607944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:00:40.607970 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:00:40.608400 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.608411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.609652 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.609660 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.611402 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:00:40.611417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.612643 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.612647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:00:40.612911 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.612949 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:00:40.613039 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 16:00:40.613157 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":52,"thread_start_us":428,"threads_started":1}
I20260928 16:00:40.614714 6603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5431ea3664e4da187b6be66a3cfb395" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9affaea8f0094261be224a439969eca0" is_pre_election: true
I20260928 16:00:40.614730 6335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5431ea3664e4da187b6be66a3cfb395" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a46693e0f9430b85f4df38e8375138" is_pre_election: true
I20260928 16:00:40.615183 6603 raft_consensus.cc:2454] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 0.
I20260928 16:00:40.615209 6335 raft_consensus.cc:2454] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 0.
I20260928 16:00:40.615815 6405 leader_election.cc:302] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0; no voters:
I20260928 16:00:40.616247 6673 raft_consensus.cc:2781] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:00:40.616396 6673 raft_consensus.cc:482] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:00:40.616504 6673 raft_consensus.cc:3037] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:40.618567 6673 raft_consensus.cc:504] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.619155 6673 leader_election.cc:288] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 election: Requested vote from peers 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495), 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851)
I20260928 16:00:40.619477 6335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5431ea3664e4da187b6be66a3cfb395" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a46693e0f9430b85f4df38e8375138"
I20260928 16:00:40.619665 6335 raft_consensus.cc:3037] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:40.619663 6603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5431ea3664e4da187b6be66a3cfb395" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9affaea8f0094261be224a439969eca0"
I20260928 16:00:40.619860 6603 raft_consensus.cc:3037] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:00:40.622270 6603 raft_consensus.cc:2454] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 1.
I20260928 16:00:40.622272 6335 raft_consensus.cc:2454] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 1.
I20260928 16:00:40.622637 6405 leader_election.cc:302] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0; no voters:
I20260928 16:00:40.622939 6673 raft_consensus.cc:2781] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:00:40.623399 6673 raft_consensus.cc:686] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 1 LEADER]: Becoming Leader. State: Replica: 76093d4c30d44531bfaa251d001c881f, State: Running, Role: LEADER
I20260928 16:00:40.623857 6673 consensus_queue.cc:237] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } }
I20260928 16:00:40.628458 6190 catalog_manager.cc:5495] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f reported cstate change: term changed from 0 to 1, leader changed from <none> to 76093d4c30d44531bfaa251d001c881f (127.22.125.194). New cstate: current_term: 1 leader_uuid: "76093d4c30d44531bfaa251d001c881f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } health_report { overall_health: UNKNOWN } } }
I20260928 16:00:40.645577 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.640146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55084 (local address 127.22.125.194:39101)
0928 16:00:40.640357 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.640363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.640452 (+ 89us) server_negotiation.cc:407] Connection header received
0928 16:00:40.640672 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.640678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.640760 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.640908 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:00:40.641301 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.642023 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.643472 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.643876 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.644010 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.644529 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:00:40.644983 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:00:40.645156 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.645210 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.645284 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:00:40.653393 6603 raft_consensus.cc:1261] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Refusing update from remote peer 76093d4c30d44531bfaa251d001c881f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:40.653393 6335 raft_consensus.cc:1261] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Refusing update from remote peer 76093d4c30d44531bfaa251d001c881f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:00:40.654590 6677 consensus_queue.cc:1035] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:40.654968 6673 consensus_queue.cc:1035] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [LEADER]: Connected to new peer: Peer: permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:00:40.667127 6680 mvcc.cc:205] Tried to move back new op lower bound from 7334343641705828352 to 7334343641598816256. Current Snapshot: MvccSnapshot[applied={T|T < 7334343641705828352}]
I20260928 16:00:40.681835 6190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:40.682120 6190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:00:40.686070 6190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Sending DeleteTablet for 3 replicas of tablet a5431ea3664e4da187b6be66a3cfb395
I20260928 16:00:40.687425 6450 tablet_service.cc:1514] Processing DeleteTablet for tablet a5431ea3664e4da187b6be66a3cfb395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:40 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:55072
I20260928 16:00:40.687525 6315 tablet_service.cc:1514] Processing DeleteTablet for tablet a5431ea3664e4da187b6be66a3cfb395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:40 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:50400
I20260928 16:00:40.688069 6583 tablet_service.cc:1514] Processing DeleteTablet for tablet a5431ea3664e4da187b6be66a3cfb395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:00:40 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:45544
I20260928 16:00:40.690810 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 6252
I20260928 16:00:40.712594 6691 tablet_replica.cc:330] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: stopping tablet replica
I20260928 16:00:40.713821 6689 tablet_replica.cc:330] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: stopping tablet replica
I20260928 16:00:40.715096 6690 tablet_replica.cc:330] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: stopping tablet replica
I20260928 16:00:40.715068 6691 raft_consensus.cc:2229] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:40.715641 6691 raft_consensus.cc:2258] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:40.715611 6690 raft_consensus.cc:2229] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:00:40.716207 6690 raft_consensus.cc:2258] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:40.716390 6689 raft_consensus.cc:2229] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:00:40.717622 6689 raft_consensus.cc:2258] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:00:40.718730 6690 ts_tablet_manager.cc:1905] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:40.719195 6691 ts_tablet_manager.cc:1905] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:00:40.725523 6689 ts_tablet_manager.cc:1905] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:00:40 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 16:00:40.726441 6690 ts_tablet_manager.cc:1918] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:40.726679 6690 log.cc:1196] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/a5431ea3664e4da187b6be66a3cfb395
I20260928 16:00:40.727269 6690 ts_tablet_manager.cc:1939] T a5431ea3664e4da187b6be66a3cfb395 P 26a46693e0f9430b85f4df38e8375138: Deleting consensus metadata
I20260928 16:00:40.729239 6175 catalog_manager.cc:4841] TS 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495): tablet a5431ea3664e4da187b6be66a3cfb395 (table test-table [id=52be0074fac1499ba69821c5302b7594]) successfully deleted
I20260928 16:00:40.731737 6691 ts_tablet_manager.cc:1918] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:40.731978 6691 log.cc:1196] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/a5431ea3664e4da187b6be66a3cfb395
I20260928 16:00:40.732565 6691 ts_tablet_manager.cc:1939] T a5431ea3664e4da187b6be66a3cfb395 P 9affaea8f0094261be224a439969eca0: Deleting consensus metadata
I20260928 16:00:40.733002 6689 ts_tablet_manager.cc:1918] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:00:40.733222 6689 log.cc:1196] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/a5431ea3664e4da187b6be66a3cfb395
I20260928 16:00:40.733842 6689 ts_tablet_manager.cc:1939] T a5431ea3664e4da187b6be66a3cfb395 P 76093d4c30d44531bfaa251d001c881f: Deleting consensus metadata
I20260928 16:00:40.734465 6178 catalog_manager.cc:4841] TS 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851): tablet a5431ea3664e4da187b6be66a3cfb395 (table test-table [id=52be0074fac1499ba69821c5302b7594]) successfully deleted
I20260928 16:00:40.735255 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.692189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.22.125.193:38495)
0928 16:00:40.692436 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.692445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.714351 (+ 21906us) server_negotiation.cc:407] Connection header received
0928 16:00:40.714600 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.714610 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.714705 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.714844 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:40.717294 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.718469 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.720801 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.721469 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.721570 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.727571 (+ 6001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:40.727588 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:40.727606 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:40.727656 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:40.731605 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.732555 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.732565 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.732568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.732733 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.733213 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.733219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.733222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.733674 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:40.733975 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.734616 (+ 641us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.734852 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 16:00:40.735447 6175 catalog_manager.cc:4841] TS 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101): tablet a5431ea3664e4da187b6be66a3cfb395 (table test-table [id=52be0074fac1499ba69821c5302b7594]) successfully deleted
I20260928 16:00:40.736641 6375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:40.828379 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 6252
I20260928 16:00:40.853870 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 6386
Sep 28 16:00:40 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 16:00:40.870978 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:40.854645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.22.125.194:39101)
0928 16:00:40.854837 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:40.854844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:40.854928 (+ 84us) server_negotiation.cc:407] Connection header received
0928 16:00:40.855133 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:40.855139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:40.855212 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:40.855320 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:40.856705 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.857565 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.858865 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:40.859316 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:40.859457 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:40.864029 (+ 4572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:40.864067 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:40.864086 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:40.864140 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:40.868034 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.868732 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.868743 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.868752 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.868918 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:40.869279 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:40.869285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:40.869288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:40.869783 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:40.870144 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:40.870356 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 16:00:40.870574 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:00:40.871883 6510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:40.985148 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 6386
I20260928 16:00:41.004190 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 6520
Sep 28 16:00:41 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 16:00:41.020188 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:00:41.004962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.22.125.195:46851)
0928 16:00:41.005146 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 16:00:41.005155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:00:41.005214 (+ 59us) server_negotiation.cc:407] Connection header received
0928 16:00:41.005440 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:00:41.005444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:00:41.005518 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:00:41.005625 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:00:41.006983 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:41.007694 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:41.009179 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:00:41.009571 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:00:41.009710 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:00:41.014359 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:00:41.014373 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:00:41.014394 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:00:41.014428 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:00:41.017109 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:41.017809 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:41.017816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:41.017818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:41.017978 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:00:41.018323 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:00:41.018326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:00:41.018328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:00:41.018739 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:00:41.019059 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0928 16:00:41.019486 (+ 427us) server_negotiation.cc:299] Negotiation successful
0928 16:00:41.019755 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:00:41.020812 6643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:41.109608 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 6520
I20260928 16:00:41.126353 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 6159
I20260928 16:00:41.127496 6221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:00:41.239573 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 6159
I20260928 16:01:01.261121 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45811
--webserver_interface=127.22.125.254
--webserver_port=36361
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:45811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:01.594803 6697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:01.595098 6697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:01.595289 6697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:01.602615 6697 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 16:01:01.602702 6697 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 16:01:01.602794 6697 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 16:01:01.602890 6697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:01:01.602989 6697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:01.603125 6697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:01.603178 6697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:01:01.603283 6697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:01:01.613703 6697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:45811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=36361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:01.615032 6697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:01.616390 6697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:01.640118 6706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:01.640121 6703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:01.640105 6704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:01.645430 6697 server_base.cc:1034] running on GCE node
I20260928 16:01:01.646271 6697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:01.647881 6697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:01.649109 6697 hybrid_clock.cc:648] HybridClock initialized: now 1790611261649065 us; error 51 us; skew 500 ppm
I20260928 16:01:01.649473 6697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:01 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611261, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:01.660935 6697 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:01:01.662667 6697 webserver.cc:460] Webserver started at http://127.22.125.254:36361/ using document root <none> and password file <none>
I20260928 16:01:01.663511 6697 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:01.663611 6697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:01.669658 6697 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 16:01:01.672554 6713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:01.673907 6697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:01.674086 6697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "91f34e2a31b44ef89b5297d57c752433"
format_stamp: "Formatted at 2026-09-28 16:00:38 on dist-test-slave-9cqd"
server_key: "2eb02a56f29595e023813d3406dd9ce3"
server_key_iv: "9408bb3d9802b1f33aed9e3db3a44a55"
server_key_version: "encryptionkey@0"
I20260928 16:01:01.675151 6697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:01.713080 6697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:01.717454 6697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:01.717770 6697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:01.736776 6697 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:45811
I20260928 16:01:01.736800 6764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:45811 every 8 connection(s)
I20260928 16:01:01.738322 6697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:01:01.745235 6765 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:01:01.747255 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 6697
I20260928 16:01:01.748097 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:38495
--local_ip_for_outbound_sockets=127.22.125.193
--tserver_master_addrs=127.22.125.254:45811
--webserver_port=37853
--webserver_interface=127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:01:01.748714 6765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Bootstrap starting.
I20260928 16:01:01.768056 6765 log.cc:824] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:01.780119 6765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:01:01.780619 6765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Bootstrap complete.
I20260928 16:01:01.789793 6765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:01.790632 6765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f34e2a31b44ef89b5297d57c752433, State: Initialized, Role: FOLLOWER
I20260928 16:01:01.791203 6765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:01.791474 6765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:01:01.791602 6765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:01:01.791772 6765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 1 FOLLOWER]: Advancing to term 2
I20260928 16:01:01.795504 6765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:01.796021 6765 leader_election.cc:302] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91f34e2a31b44ef89b5297d57c752433; no voters:
I20260928 16:01:01.796706 6765 leader_election.cc:288] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 16:01:01.796840 6769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 16:01:01.798305 6769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 LEADER]: Becoming Leader. State: Replica: 91f34e2a31b44ef89b5297d57c752433, State: Running, Role: LEADER
I20260928 16:01:01.799067 6769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:01.799252 6765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:01:01.801783 6771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91f34e2a31b44ef89b5297d57c752433. Latest consensus state: current_term: 2 leader_uuid: "91f34e2a31b44ef89b5297d57c752433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } } }
I20260928 16:01:01.802225 6771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:01.802770 6770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "91f34e2a31b44ef89b5297d57c752433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } } }
I20260928 16:01:01.803114 6770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:01.813342 6775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:01:01.821856 6775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:01:01.823256 6775 catalog_manager.cc:1259] Loaded cluster ID: 949761a23cc24b3ca723e920706c03d8
I20260928 16:01:01.823423 6775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:01:01.827595 6775 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:01:01.829330 6775 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Loaded TSK: 0
I20260928 16:01:01.855544 6786 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 16:01:01.860954 6775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Generated new TSK 1
I20260928 16:01:01.861431 6775 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 16:01:02.107993 6767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:02.108352 6767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:02.108506 6767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:02.115885 6767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:02.116086 6767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:02.116276 6767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:01:02.126134 6767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:38495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=37853
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:02.127430 6767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:02.128815 6767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:02.143509 6793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:02.143949 6792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:02.143972 6795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:02.145692 6767 server_base.cc:1034] running on GCE node
I20260928 16:01:02.146447 6767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:02.147745 6767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:02.149029 6767 hybrid_clock.cc:648] HybridClock initialized: now 1790611262148942 us; error 76 us; skew 500 ppm
I20260928 16:01:02.149447 6767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:02 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611262, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:02.153420 6767 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:01:02.154994 6767 webserver.cc:460] Webserver started at http://127.22.125.193:37853/ using document root <none> and password file <none>
I20260928 16:01:02.155720 6767 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:02.155859 6767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:02.161480 6767 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260928 16:01:02.164436 6802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:02.165819 6767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:02.166015 6767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "26a46693e0f9430b85f4df38e8375138"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "4a5fa37c917ec736af6f6212edc0672e"
server_key_iv: "24f930e0d7f058a4a66ff43862f726b6"
server_key_version: "encryptionkey@0"
I20260928 16:01:02.166985 6767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:02.188313 6767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:02.192898 6767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:02.193186 6767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:02.194223 6767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:02.195704 6767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:02.195851 6767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:02.195981 6767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:02.196084 6767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:02.229589 6767 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:38495
I20260928 16:01:02.229645 6914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:38495 every 8 connection(s)
I20260928 16:01:02.231031 6767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:01:02.241201 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 6767
I20260928 16:01:02.241904 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:39101
--local_ip_for_outbound_sockets=127.22.125.194
--tserver_master_addrs=127.22.125.254:45811
--webserver_port=35113
--webserver_interface=127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:01:02 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611262, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:02.254343 6918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:02.234131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:43277)
0928 16:01:02.234945 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:02.234993 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:02.236835 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:02.238046 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:02.238053 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:02.238569 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:02.239102 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:02.239118 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.240800 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.240811 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:02.242161 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:02.242178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.243215 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.243223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:02.243509 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:02.244207 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:02.244228 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:02.246553 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:02.250115 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:02.250126 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:02.250139 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:02.250558 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:02.250997 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:02.251004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:02.251007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:02.251196 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:02.251923 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:02.251932 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:02.252294 (+ 362us) client_negotiation.cc:769] Sending connection context
0928 16:01:02.252557 (+ 263us) client_negotiation.cc:240] Negotiation successful
0928 16:01:02.253084 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":343,"threads_started":1}
I20260928 16:01:02.254359 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:02.234003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:43277 (local address 127.22.125.254:45811)
0928 16:01:02.234655 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:02.234667 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:02.235122 (+ 455us) server_negotiation.cc:407] Connection header received
0928 16:01:02.237300 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:02.237314 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:02.237627 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:02.237923 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:02.239297 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.240632 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.242323 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.242967 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.243197 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:02.246810 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:02.246818 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:02.246834 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:02.246875 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:02.249965 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:02.250700 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:02.250713 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:02.250718 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:02.250879 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:02.251312 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:02.251318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:02.251321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:02.251764 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:02.251981 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:02.252773 (+ 792us) server_negotiation.cc:299] Negotiation successful
0928 16:01:02.253142 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":295,"threads_started":1}
I20260928 16:01:02.257181 6915 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45811
I20260928 16:01:02.257514 6915 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:02.258399 6915 heartbeater.cc:511] Master 127.22.125.254:45811 requested a full tablet report, sending...
I20260928 16:01:02.260825 6730 ts_manager.cc:194] Registered new tserver with Master: 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495)
I20260928 16:01:02.262287 6730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:43277
W20260928 16:01:02.575449 6919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:02.575706 6919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:02.575798 6919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:02.582952 6919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:02.583086 6919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:02.583269 6919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:01:02.592594 6919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:39101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=35113
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:02.593717 6919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:02.595002 6919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:02.608021 6925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:02.608069 6924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:02.608676 6927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:02.609140 6919 server_base.cc:1034] running on GCE node
I20260928 16:01:02.609709 6919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:02.610513 6919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:02.611738 6919 hybrid_clock.cc:648] HybridClock initialized: now 1790611262611701 us; error 49 us; skew 500 ppm
I20260928 16:01:02.612120 6919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:02 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611262, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:02.615295 6919 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:01:02.616712 6919 webserver.cc:460] Webserver started at http://127.22.125.194:35113/ using document root <none> and password file <none>
I20260928 16:01:02.617344 6919 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:02.617504 6919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:02.622424 6919 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260928 16:01:02.624974 6934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:02.626226 6919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:02.626398 6919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "76093d4c30d44531bfaa251d001c881f"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "94f205a2e968bc1374dba28e98b67c5d"
server_key_iv: "52a1b6814657fda5b69acf17251d0e97"
server_key_version: "encryptionkey@0"
I20260928 16:01:02.627316 6919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:02.655148 6919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:02.659206 6919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:02.659497 6919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:02.660452 6919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:02.661725 6919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:02.661861 6919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:02.661990 6919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:02.662076 6919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:02.694927 6919 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:39101
I20260928 16:01:02.694983 7046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:39101 every 8 connection(s)
I20260928 16:01:02.696424 6919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:01:02.705579 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 6919
I20260928 16:01:02.706367 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:46851
--local_ip_for_outbound_sockets=127.22.125.195
--tserver_master_addrs=127.22.125.254:45811
--webserver_port=40577
--webserver_interface=127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 16:01:02 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611262, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:02.716996 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:02.698875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:58643 (local address 127.22.125.254:45811)
0928 16:01:02.699065 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:02.699070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:02.700318 (+ 1248us) server_negotiation.cc:407] Connection header received
0928 16:01:02.702098 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:02.702104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:02.702193 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:02.702303 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:02.703769 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.704544 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.706718 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.707134 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.707232 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:02.711280 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:02.711285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:02.711288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:02.711324 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:02.713965 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:02.714672 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:02.714680 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:02.714683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:02.714809 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:02.715238 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:02.715244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:02.715248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:02.715525 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:02.715685 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:02.716385 (+ 700us) server_negotiation.cc:299] Negotiation successful
0928 16:01:02.716614 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:01:02.717888 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:02.699381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:58643)
0928 16:01:02.700156 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:02.700205 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:02.701840 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:02.702534 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:02.702541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:02.703055 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:02.703594 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:02.703611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.704696 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.704708 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:02.706564 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:02.706582 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:02.707364 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:02.707374 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:02.707705 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:02.708461 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:02.708514 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:02.711058 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:02.714096 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:02.714105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:02.714119 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:02.714530 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:02.714936 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:02.714941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:02.714945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:02.715122 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:02.715661 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:02.715668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:02.715975 (+ 307us) client_negotiation.cc:769] Sending connection context
0928 16:01:02.716297 (+ 322us) client_negotiation.cc:240] Negotiation successful
0928 16:01:02.716808 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":309,"threads_started":1}
I20260928 16:01:02.719303 7047 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45811
I20260928 16:01:02.719609 7047 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:02.720340 7047 heartbeater.cc:511] Master 127.22.125.254:45811 requested a full tablet report, sending...
I20260928 16:01:02.721973 6730 ts_manager.cc:194] Registered new tserver with Master: 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101)
I20260928 16:01:02.723270 6730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:58643
W20260928 16:01:03.029838 7050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:03.030138 7050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:03.030260 7050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:03.037530 7050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:03.037626 7050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:03.037741 7050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:01:03.046924 7050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:46851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=40577
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:03.048123 7050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:03.049491 7050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:03.062239 7056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:03.062285 7058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:03.062251 7055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:03.063030 7050 server_base.cc:1034] running on GCE node
I20260928 16:01:03.063707 7050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:03.064684 7050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:03.065881 7050 hybrid_clock.cc:648] HybridClock initialized: now 1790611263065832 us; error 56 us; skew 500 ppm
I20260928 16:01:03.066195 7050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:03 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611263, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:03.069224 7050 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:01:03.070653 7050 webserver.cc:460] Webserver started at http://127.22.125.195:40577/ using document root <none> and password file <none>
I20260928 16:01:03.071483 7050 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:03.071637 7050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:03.077279 7050 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260928 16:01:03.080109 7065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:03.081391 7050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:03.081672 7050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "9affaea8f0094261be224a439969eca0"
format_stamp: "Formatted at 2026-09-28 16:00:40 on dist-test-slave-9cqd"
server_key: "9b9fd5f5ed5cefb7950e7907c0aad765"
server_key_iv: "111326e18386bcd8ad90c8b1d4eb19b1"
server_key_version: "encryptionkey@0"
I20260928 16:01:03.082649 7050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:03.097151 7050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:03.101068 7050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:03.101334 7050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:03.102279 7050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:03.103549 7050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:03.103667 7050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:03.103793 7050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:03.103888 7050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:03.137144 7050 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:46851
I20260928 16:01:03.137197 7177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:46851 every 8 connection(s)
I20260928 16:01:03.138759 7050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:01:03.141014 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 7050
Sep 28 16:01:03 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611263, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:03.158618 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.142057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.22.125.254:45811)
0928 16:01:03.145037 (+ 2980us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.145043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.145059 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:01:03.145121 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.145128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.145209 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.145335 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:03.146913 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.147681 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.149162 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.149775 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.149912 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.152654 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:03.152661 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:03.152667 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:03.152703 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:03.155748 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.156506 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.156512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.156515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.156663 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.157079 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.157083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.157085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.157362 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:03.157664 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.158011 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.158238 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2880,"thread_start_us":2749,"threads_started":1}
I20260928 16:01:03.161221 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.141538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:53897 (local address 127.22.125.254:45811)
0928 16:01:03.141762 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.141771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.144695 (+ 2924us) server_negotiation.cc:407] Connection header received
0928 16:01:03.146727 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.146732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.146804 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.146919 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:03.148451 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.149182 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.150987 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.151405 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.151531 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.155220 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:03.155225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:03.155228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:03.155266 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:03.157875 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.158659 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.158684 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.158690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.158850 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.159390 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.159395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.159397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.159670 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:03.159821 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.160654 (+ 833us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.160898 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:01:03.162128 7182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.142146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:53897)
0928 16:01:03.144544 (+ 2398us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.144586 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:03.146474 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:03.147158 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:03.147166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:03.147659 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:03.148292 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.148311 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.149349 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.149357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.150845 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.150856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.151634 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.151642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.151867 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.152529 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:03.152551 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:03.155029 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:03.158029 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:03.158038 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:03.158053 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:03.158541 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:03.158974 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:03.158979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:03.158982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:03.159211 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:03.159824 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:03.159833 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:03.160284 (+ 451us) client_negotiation.cc:769] Sending connection context
0928 16:01:03.160536 (+ 252us) client_negotiation.cc:240] Negotiation successful
0928 16:01:03.161053 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2153,"thread_start_us":1945,"threads_started":1}
I20260928 16:01:03.163424 7178 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45811
I20260928 16:01:03.163688 7178 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:03.164467 7178 heartbeater.cc:511] Master 127.22.125.254:45811 requested a full tablet report, sending...
I20260928 16:01:03.165764 6730 ts_manager.cc:194] Registered new tserver with Master: 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851)
I20260928 16:01:03.166604 6730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:53897
I20260928 16:01:03.173367 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:01:03.179788 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.174330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.22.125.254:45811)
0928 16:01:03.174480 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.174484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.174852 (+ 368us) server_negotiation.cc:407] Connection header received
0928 16:01:03.175023 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.175027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.175150 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.175262 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:01:03.175743 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.176514 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.178107 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.178520 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.178629 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.179091 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:01:03.179285 (+ 194us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 16:01:03.179558 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57928: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260928 16:01:03.179910 6917 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57928: token expired
W20260928 16:01:03.180485 6653 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.125.254:45811)
I20260928 16:01:03.192901 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.181095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.22.125.254:45811)
0928 16:01:03.181235 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.181238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.181253 (+ 15us) server_negotiation.cc:407] Connection header received
0928 16:01:03.181288 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.181292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.181342 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.181416 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:03.182774 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.183436 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.184790 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.185173 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.185261 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.187869 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:03.187874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:03.187877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:03.187906 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:03.190569 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.191151 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.191154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.191157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.191268 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.191582 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.191585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.191587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.191900 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:03.192148 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.192497 (+ 349us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.192659 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 16:01:03.196247 6730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:01:03.203552 6730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:01:03.227001 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.214822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.22.125.195:46851)
0928 16:01:03.216684 (+ 1862us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.216695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.216736 (+ 41us) server_negotiation.cc:407] Connection header received
0928 16:01:03.216862 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.216873 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.217175 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.217427 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:03.218433 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.220926 (+ 2493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.224838 (+ 3912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.225748 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.225976 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.226180 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.226478 (+ 298us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.226635 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1646,"thread_start_us":1459,"threads_started":1}
I20260928 16:01:03.227065 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.214555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:46851 (local address 127.0.0.1:48536)
0928 16:01:03.215494 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.215540 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:03.215738 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:03.217488 (+ 1750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:03.217499 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:03.217544 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:03.218150 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.218169 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.221125 (+ 2956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.221134 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.224687 (+ 3553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.224699 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.225988 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.225994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.226228 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.226273 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 16:01:03.226429 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 16:01:03.226619 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":1153,"threads_started":1}
I20260928 16:01:03.231506 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.214634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:39101 (local address 127.0.0.1:46062)
0928 16:01:03.218218 (+ 3584us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.218237 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:03.218359 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:03.223835 (+ 5476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:03.223840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:03.223863 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:03.224309 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.224339 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.225970 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.225979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.228473 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.228488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.230458 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.230464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.230651 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.230694 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 16:01:03.230765 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 16:01:03.230983 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3437,"mutex_wait_us":2,"thread_start_us":3255,"threads_started":1}
I20260928 16:01:03.232502 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.215493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.22.125.194:39101)
0928 16:01:03.223208 (+ 7715us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.223215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.223242 (+ 27us) server_negotiation.cc:407] Connection header received
0928 16:01:03.223357 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.223364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.223614 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.224591 (+ 977us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:03.224849 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.225823 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.228657 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.230164 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.231757 (+ 1593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.231935 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.232022 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.232219 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7564,"thread_start_us":7456,"threads_started":1}
I20260928 16:01:03.231564 7113 tablet_service.cc:1467] Processing CreateTablet for tablet 312171ef81c141b0b767a1b9f3dfd4f0 (DEFAULT_TABLE table=test-table [id=00fd8000278645febb28b69e575e9178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:03.233982 7113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312171ef81c141b0b767a1b9f3dfd4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:03.234179 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.219147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45346)
0928 16:01:03.224687 (+ 5540us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.224704 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:03.224833 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:03.226822 (+ 1989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:03.226828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:03.226859 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:03.227248 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.227260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.229182 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.229188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.231338 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.231353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.233348 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.233354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.233529 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.233567 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 16:01:03.233669 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 16:01:03.233779 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5379,"mutex_wait_us":83,"thread_start_us":5230,"threads_started":1}
I20260928 16:01:03.234952 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.218573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.22.125.193:38495)
0928 16:01:03.219358 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.219366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.226017 (+ 6651us) server_negotiation.cc:407] Connection header received
0928 16:01:03.226141 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.226151 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.226389 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.226568 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:03.227444 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.229006 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.231547 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.233068 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.233303 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.233480 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.234384 (+ 904us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.234557 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":706,"threads_started":1}
I20260928 16:01:03.237154 6982 tablet_service.cc:1467] Processing CreateTablet for tablet 312171ef81c141b0b767a1b9f3dfd4f0 (DEFAULT_TABLE table=test-table [id=00fd8000278645febb28b69e575e9178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:03.238355 6982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312171ef81c141b0b767a1b9f3dfd4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:03.238608 6850 tablet_service.cc:1467] Processing CreateTablet for tablet 312171ef81c141b0b767a1b9f3dfd4f0 (DEFAULT_TABLE table=test-table [id=00fd8000278645febb28b69e575e9178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:03.239930 6850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312171ef81c141b0b767a1b9f3dfd4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:03.249300 7194 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Bootstrap starting.
I20260928 16:01:03.251472 7195 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Bootstrap starting.
I20260928 16:01:03.253072 7194 tablet_bootstrap.cc:654] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:03.254369 7194 log.cc:824] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:03.254959 7195 tablet_bootstrap.cc:654] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:03.256659 7195 log.cc:824] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:03.258955 7196 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Bootstrap starting.
I20260928 16:01:03.260200 7195 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: No bootstrap required, opened a new log
I20260928 16:01:03.260489 7195 ts_tablet_manager.cc:1397] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260928 16:01:03.260804 7194 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: No bootstrap required, opened a new log
I20260928 16:01:03.261055 7194 ts_tablet_manager.cc:1397] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260928 16:01:03.262274 7196 tablet_bootstrap.cc:654] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:03.263420 7196 log.cc:824] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:03.266287 6915 heartbeater.cc:503] Master 127.22.125.254:45811 was elected leader, sending a full tablet report...
I20260928 16:01:03.266515 7196 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: No bootstrap required, opened a new log
I20260928 16:01:03.266749 7196 ts_tablet_manager.cc:1397] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 16:01:03.270176 7194 raft_consensus.cc:348] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.270730 7194 raft_consensus.cc:374] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:03.270924 7194 raft_consensus.cc:729] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76093d4c30d44531bfaa251d001c881f, State: Initialized, Role: FOLLOWER
I20260928 16:01:03.270957 7195 raft_consensus.cc:348] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.271443 7195 raft_consensus.cc:374] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:03.271545 7195 raft_consensus.cc:729] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9affaea8f0094261be224a439969eca0, State: Initialized, Role: FOLLOWER
I20260928 16:01:03.271641 7194 consensus_queue.cc:260] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.272125 7195 consensus_queue.cc:260] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.272830 7047 heartbeater.cc:503] Master 127.22.125.254:45811 was elected leader, sending a full tablet report...
I20260928 16:01:03.273404 7194 ts_tablet_manager.cc:1428] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260928 16:01:03.276713 7178 heartbeater.cc:503] Master 127.22.125.254:45811 was elected leader, sending a full tablet report...
I20260928 16:01:03.277299 7195 ts_tablet_manager.cc:1428] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260928 16:01:03.278496 7196 raft_consensus.cc:348] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.278823 7196 raft_consensus.cc:374] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:03.278936 7196 raft_consensus.cc:729] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a46693e0f9430b85f4df38e8375138, State: Initialized, Role: FOLLOWER
I20260928 16:01:03.279547 7196 consensus_queue.cc:260] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.281064 7196 ts_tablet_manager.cc:1428] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Time spent starting tablet: real 0.014s user 0.008s sys 0.006s
W20260928 16:01:03.390389 7179 tablet.cc:2369] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:01:03.448518 7048 tablet.cc:2369] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:01:03.453168 7200 raft_consensus.cc:482] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:03.453452 7200 raft_consensus.cc:504] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.455029 7200 leader_election.cc:288] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851), 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495)
I20260928 16:01:03.463833 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.455637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:50079 (local address 127.22.125.193:38495)
0928 16:01:03.455852 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.455859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.456736 (+ 877us) server_negotiation.cc:407] Connection header received
0928 16:01:03.456925 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.456931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.457015 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.457108 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:03.457762 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.458826 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.461444 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.462674 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.462846 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.462920 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.463444 (+ 524us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.463573 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 16:01:03.463848 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.455647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:38623 (local address 127.22.125.195:46851)
0928 16:01:03.455897 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.455905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.456546 (+ 641us) server_negotiation.cc:407] Connection header received
0928 16:01:03.456774 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.456778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.456837 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.456934 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:03.457483 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.458600 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.461445 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.462676 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.462857 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.462920 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.463444 (+ 524us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.463573 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 16:01:03.464114 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.455630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:46851 (local address 127.22.125.194:38623)
0928 16:01:03.456431 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.456447 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:03.456620 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:03.456948 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:03.456953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:03.456975 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:03.457325 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.457338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.458735 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.458738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.461287 (+ 2549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.461302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.462938 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.462945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.463174 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.463304 (+ 130us) client_negotiation.cc:769] Sending connection context
0928 16:01:03.463401 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 16:01:03.463634 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":138,"thread_start_us":458,"threads_started":1}
I20260928 16:01:03.464506 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.455643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.22.125.194:50079)
0928 16:01:03.456627 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.456641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:03.456784 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:03.457165 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:03.457170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:03.457202 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:03.457614 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.457623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.458962 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.458969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.461287 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:03.461302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.462941 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.462946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:03.463174 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.463307 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 16:01:03.463401 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 16:01:03.464093 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":666,"thread_start_us":665,"threads_started":1}
I20260928 16:01:03.465112 7133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312171ef81c141b0b767a1b9f3dfd4f0" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9affaea8f0094261be224a439969eca0" is_pre_election: true
I20260928 16:01:03.465584 7133 raft_consensus.cc:2454] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 0.
I20260928 16:01:03.465513 6870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312171ef81c141b0b767a1b9f3dfd4f0" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a46693e0f9430b85f4df38e8375138" is_pre_election: true
I20260928 16:01:03.466002 6870 raft_consensus.cc:2454] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 0.
I20260928 16:01:03.466279 6938 leader_election.cc:302] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0; no voters:
I20260928 16:01:03.466666 7200 raft_consensus.cc:2781] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:01:03.466796 7200 raft_consensus.cc:482] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:01:03.466872 7200 raft_consensus.cc:3037] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:03.469040 7200 raft_consensus.cc:504] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.469645 7200 leader_election.cc:288] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 election: Requested vote from peers 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851), 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495)
I20260928 16:01:03.470079 7133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312171ef81c141b0b767a1b9f3dfd4f0" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9affaea8f0094261be224a439969eca0"
I20260928 16:01:03.470093 6870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312171ef81c141b0b767a1b9f3dfd4f0" candidate_uuid: "76093d4c30d44531bfaa251d001c881f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a46693e0f9430b85f4df38e8375138"
I20260928 16:01:03.470287 7133 raft_consensus.cc:3037] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:03.470291 6870 raft_consensus.cc:3037] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:03.472702 7133 raft_consensus.cc:2454] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 1.
I20260928 16:01:03.472713 6870 raft_consensus.cc:2454] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76093d4c30d44531bfaa251d001c881f in term 1.
I20260928 16:01:03.473168 6938 leader_election.cc:302] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0; no voters:
I20260928 16:01:03.473471 7200 raft_consensus.cc:2781] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:03.473910 7200 raft_consensus.cc:686] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 1 LEADER]: Becoming Leader. State: Replica: 76093d4c30d44531bfaa251d001c881f, State: Running, Role: LEADER
I20260928 16:01:03.474372 7200 consensus_queue.cc:237] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:03.479166 6729 catalog_manager.cc:5495] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f reported cstate change: term changed from 0 to 1, leader changed from <none> to 76093d4c30d44531bfaa251d001c881f (127.22.125.194). New cstate: current_term: 1 leader_uuid: "76093d4c30d44531bfaa251d001c881f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } health_report { overall_health: UNKNOWN } } }
W20260928 16:01:03.483319 6916 tablet.cc:2369] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:01:03.520339 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.514791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.22.125.194:39101)
0928 16:01:03.515028 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.515033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.515051 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:01:03.515289 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.515293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.515370 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.515555 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:01:03.515929 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.516703 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.518164 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.518576 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.518712 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.519225 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:01:03.519717 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:01:03.519890 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.519951 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.520027 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:01:03.528529 7133 raft_consensus.cc:1261] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Refusing update from remote peer 76093d4c30d44531bfaa251d001c881f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:03.528541 6870 raft_consensus.cc:1261] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Refusing update from remote peer 76093d4c30d44531bfaa251d001c881f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:03.529492 7200 consensus_queue.cc:1035] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [LEADER]: Connected to new peer: Peer: permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:03.529778 7205 consensus_queue.cc:1035] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:03.555004 6729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:03.555284 6729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:03.558682 6729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Sending DeleteTablet for 3 replicas of tablet 312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:03.559760 7113 tablet_service.cc:1514] Processing DeleteTablet for tablet 312171ef81c141b0b767a1b9f3dfd4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:03 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:48536
I20260928 16:01:03.559922 6850 tablet_service.cc:1514] Processing DeleteTablet for tablet 312171ef81c141b0b767a1b9f3dfd4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:03 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:45346
I20260928 16:01:03.559929 6982 tablet_service.cc:1514] Processing DeleteTablet for tablet 312171ef81c141b0b767a1b9f3dfd4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:03 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:46062
I20260928 16:01:03.561187 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 6767
I20260928 16:01:03.570711 7216 tablet_replica.cc:330] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: stopping tablet replica
I20260928 16:01:03.571281 7216 raft_consensus.cc:2229] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:03.571883 7216 raft_consensus.cc:2258] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:03.572746 7218 tablet_replica.cc:330] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: stopping tablet replica
I20260928 16:01:03.573148 7217 tablet_replica.cc:330] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: stopping tablet replica
I20260928 16:01:03.573628 7217 raft_consensus.cc:2229] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:01:03.574450 7217 raft_consensus.cc:2258] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:03.574805 7216 ts_tablet_manager.cc:1905] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:03.576745 7217 ts_tablet_manager.cc:1905] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:03.578835 7218 raft_consensus.cc:2229] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:03.579639 7218 raft_consensus.cc:2258] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:03.581875 7218 ts_tablet_manager.cc:1905] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:03.582902 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.562125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.22.125.193:38495)
0928 16:01:03.562276 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.562282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.562355 (+ 73us) server_negotiation.cc:407] Connection header received
0928 16:01:03.562548 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.562555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.562644 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.562736 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:03.565132 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.566650 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.569220 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.570141 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.570308 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.575313 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:03.575332 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:03.575351 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:03.575397 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:03.579219 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.580196 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.580205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.580208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.580382 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.580893 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.580898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.580901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.581413 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:03.581714 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.582296 (+ 582us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.582521 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:01:03.584196 7217 ts_tablet_manager.cc:1918] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:03.584395 7217 log.cc:1196] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:03.584651 6910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:03.585031 7217 ts_tablet_manager.cc:1939] T 312171ef81c141b0b767a1b9f3dfd4f0 P 76093d4c30d44531bfaa251d001c881f: Deleting consensus metadata
I20260928 16:01:03.586315 7216 ts_tablet_manager.cc:1918] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:03.586483 7216 log.cc:1196] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:03.587074 7216 ts_tablet_manager.cc:1939] T 312171ef81c141b0b767a1b9f3dfd4f0 P 9affaea8f0094261be224a439969eca0: Deleting consensus metadata
I20260928 16:01:03.587141 6714 catalog_manager.cc:4841] TS 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101): tablet 312171ef81c141b0b767a1b9f3dfd4f0 (table test-table [id=00fd8000278645febb28b69e575e9178]) successfully deleted
I20260928 16:01:03.588897 6717 catalog_manager.cc:4841] TS 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851): tablet 312171ef81c141b0b767a1b9f3dfd4f0 (table test-table [id=00fd8000278645febb28b69e575e9178]) successfully deleted
I20260928 16:01:03.678579 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 6767
W20260928 16:01:03.697420 6714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.125.193:38495 (error 108)
I20260928 16:01:03.699127 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.698638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45362)
0928 16:01:03.698833 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.698951 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 16:01:03.699142 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 6919
W20260928 16:01:03.699738 6714 catalog_manager.cc:4584] TS 26a46693e0f9430b85f4df38e8375138 (127.22.125.193:38495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 312171ef81c141b0b767a1b9f3dfd4f0: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
I20260928 16:01:03.714241 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.700404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46080 (local address 127.22.125.194:39101)
0928 16:01:03.700701 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.700718 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.700738 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:01:03.700879 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.700884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.700966 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.701078 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:03.702409 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.703256 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.704674 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.705138 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.705276 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.707897 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:03.707909 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:03.707932 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:03.707962 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:03.711280 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.711963 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.711972 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.711977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.712179 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.712544 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.712550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.712553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.712929 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:03.713236 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.713542 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.713758 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260928 16:01:03.714979 7042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:03.753212 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.752775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45374)
0928 16:01:03.752978 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.753088 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 16:01:03.754256 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.753922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45380)
0928 16:01:03.754076 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.754154 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 16:01:03.814888 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.814366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45388)
0928 16:01:03.814569 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.814713 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 16:01:03.816028 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.815692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45394)
0928 16:01:03.815852 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.815924 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 16:01:03.831107 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 6919
I20260928 16:01:03.850358 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 7050
I20260928 16:01:03.863010 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.851116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.22.125.195:46851)
0928 16:01:03.851275 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:03.851280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:03.851324 (+ 44us) server_negotiation.cc:407] Connection header received
0928 16:01:03.851523 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:03.851527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:03.851597 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:03.851679 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:03.853010 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.853703 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.854944 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:03.855300 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:03.855394 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:03.858022 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:03.858034 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:03.858048 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:03.858080 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:03.860645 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.861300 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.861306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.861308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.861455 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:03.861771 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:03.861774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:03.861776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:03.862083 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:03.862299 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:03.862602 (+ 303us) server_negotiation.cc:299] Negotiation successful
0928 16:01:03.862765 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 16:01:03.863634 7173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:03.896786 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.896233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45396)
0928 16:01:03.896439 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.896532 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260928 16:01:03.897964 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:03.897561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45408)
0928 16:01:03.897750 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:03.897830 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260928 16:01:03.951881 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 7050
I20260928 16:01:03.968530 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 6697
I20260928 16:01:03.969689 6760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:04.082960 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:04.082390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:38495 (local address 127.0.0.1:45410)
0928 16:01:04.082614 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:04.082753 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 16:01:04.083518 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 6697
I20260928 16:01:24.105530 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45811
--webserver_interface=127.22.125.254
--webserver_port=36361
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:45811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:24.430158 7224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:24.430423 7224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:24.430549 7224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:24.437506 7224 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 16:01:24.437623 7224 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 16:01:24.437687 7224 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 16:01:24.437790 7224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:01:24.437886 7224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:24.437987 7224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:24.438073 7224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:01:24.438153 7224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:01:24.447538 7224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:45811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=36361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:24.448832 7224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:24.450160 7224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:24.462057 7232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:24.462100 7230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:24.462129 7229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:24.462432 7224 server_base.cc:1034] running on GCE node
I20260928 16:01:24.463586 7224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:24.464983 7224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:24.466252 7224 hybrid_clock.cc:648] HybridClock initialized: now 1790611284466173 us; error 42 us; skew 500 ppm
I20260928 16:01:24.466583 7224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:24 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:24.475092 7224 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:01:24.476634 7224 webserver.cc:460] Webserver started at http://127.22.125.254:36361/ using document root <none> and password file <none>
I20260928 16:01:24.477308 7224 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:24.477451 7224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:24.483167 7224 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 16:01:24.486215 7239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:24.487703 7224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:01:24.487874 7224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "91f34e2a31b44ef89b5297d57c752433"
format_stamp: "Formatted at 2026-09-28 16:00:38 on dist-test-slave-9cqd"
server_key: "2eb02a56f29595e023813d3406dd9ce3"
server_key_iv: "9408bb3d9802b1f33aed9e3db3a44a55"
server_key_version: "encryptionkey@0"
I20260928 16:01:24.488879 7224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:24.509258 7224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:24.513145 7224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:24.513442 7224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:24.533807 7224 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:45811
I20260928 16:01:24.533842 7290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:45811 every 8 connection(s)
I20260928 16:01:24.535305 7224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:01:24.537667 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 7224
I20260928 16:01:24.538506 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:38495
--local_ip_for_outbound_sockets=127.22.125.193
--tserver_master_addrs=127.22.125.254:45811
--webserver_port=37853
--webserver_interface=127.22.125.193
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:01:24.542352 7291 sys_catalog.cc:263] Verifying existing consensus state
I20260928 16:01:24.545572 7291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Bootstrap starting.
I20260928 16:01:24.565918 7291 log.cc:824] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:24.581732 7291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:01:24.582279 7291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Bootstrap complete.
I20260928 16:01:24.591290 7291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:24.592312 7291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f34e2a31b44ef89b5297d57c752433, State: Initialized, Role: FOLLOWER
I20260928 16:01:24.592991 7291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:24.593271 7291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:01:24.593403 7291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:01:24.593588 7291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 2 FOLLOWER]: Advancing to term 3
I20260928 16:01:24.597484 7291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:24.597977 7291 leader_election.cc:302] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91f34e2a31b44ef89b5297d57c752433; no voters:
I20260928 16:01:24.598701 7291 leader_election.cc:288] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 16:01:24.598852 7295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 16:01:24.600140 7295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [term 3 LEADER]: Becoming Leader. State: Replica: 91f34e2a31b44ef89b5297d57c752433, State: Running, Role: LEADER
I20260928 16:01:24.600776 7295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } }
I20260928 16:01:24.600979 7291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:01:24.604156 7296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "91f34e2a31b44ef89b5297d57c752433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } } }
I20260928 16:01:24.604481 7296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:24.603703 7297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91f34e2a31b44ef89b5297d57c752433. Latest consensus state: current_term: 3 leader_uuid: "91f34e2a31b44ef89b5297d57c752433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f34e2a31b44ef89b5297d57c752433" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45811 } } }
I20260928 16:01:24.605166 7297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433 [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:24.616345 7304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:01:24.625313 7304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:01:24.626781 7304 catalog_manager.cc:1259] Loaded cluster ID: 949761a23cc24b3ca723e920706c03d8
I20260928 16:01:24.626946 7304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:01:24.631091 7304 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:01:24.632652 7304 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Loaded TSK: 0
I20260928 16:01:24.632863 7304 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 91f34e2a31b44ef89b5297d57c752433: Loaded TSK: 1
F20260928 16:01:24.634078 7304 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0b6b9149ad google::LogMessage::Fail() at ??:0
@ 0x7f0b6b91788a google::LogMessage::SendToLog() at ??:0
@ 0x7f0b6b9144de google::LogMessage::Flush() at ??:0
@ 0x7f0b6b915a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0b6f6787d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0b6f66773e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0b813431c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0b81342783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0b813ce2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f0b813ce0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f0b814185e9 std::function<>::operator()() at ??:0
@ 0x7f0b81346cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0b81344c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0b813cc610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0b813cc439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0b6e560345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0b6e56391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0b6e563799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0b6e53eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0b718186db start_thread at ??:0
@ 0x7f0b6a7ae61f clone at ??:0
W20260928 16:01:24.913805 7293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:24.914115 7293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:24.914300 7293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:24.921526 7293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:24.921669 7293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:24.921844 7293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:01:24.931512 7293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:38495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=37853
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:24.932788 7293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:24.934062 7293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:24.947129 7339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:24.947165 7342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:24.947069 7340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:24.948280 7293 server_base.cc:1034] running on GCE node
I20260928 16:01:24.948856 7293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:24.949757 7293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:24.950923 7293 hybrid_clock.cc:648] HybridClock initialized: now 1790611284950862 us; error 31 us; skew 500 ppm
I20260928 16:01:24.951316 7293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:24 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:24.954370 7293 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:01:24.955812 7293 webserver.cc:460] Webserver started at http://127.22.125.193:37853/ using document root <none> and password file <none>
I20260928 16:01:24.956555 7293 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:24.956707 7293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:24.961848 7293 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 16:01:24.964884 7349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:24.966127 7293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:01:24.966310 7293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "26a46693e0f9430b85f4df38e8375138"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "4a5fa37c917ec736af6f6212edc0672e"
server_key_iv: "24f930e0d7f058a4a66ff43862f726b6"
server_key_version: "encryptionkey@0"
I20260928 16:01:24.967238 7293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:24.984304 7293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:24.988129 7293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:24.988404 7293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:24.989362 7293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:24.991072 7356 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260928 16:01:24.996439 7293 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 16:01:24.996587 7293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260928 16:01:24.996722 7293 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 16:01:24.999737 7293 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 16:01:24.999847 7293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260928 16:01:25.000248 7356 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Bootstrap starting.
I20260928 16:01:25.008432 7356 log.cc:824] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:25.022408 7356 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 16:01:25.023015 7356 tablet_bootstrap.cc:492] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Bootstrap complete.
I20260928 16:01:25.023561 7356 ts_tablet_manager.cc:1397] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Time spent bootstrapping tablet: real 0.024s user 0.013s sys 0.006s
I20260928 16:01:25.034072 7356 raft_consensus.cc:348] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:25.035712 7356 raft_consensus.cc:729] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a46693e0f9430b85f4df38e8375138, State: Initialized, Role: FOLLOWER
I20260928 16:01:25.036574 7356 consensus_queue.cc:260] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:25.038039 7356 ts_tablet_manager.cc:1428] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138: Time spent starting tablet: real 0.014s user 0.012s sys 0.005s
I20260928 16:01:25.046304 7293 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:38495
I20260928 16:01:25.046348 7464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:38495 every 8 connection(s)
I20260928 16:01:25.047513 7293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:01:25.051962 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.050205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:36105)
0928 16:01:25.050947 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.051163 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":306,"threads_started":1}
W20260928 16:01:25.052623 7465 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:45811 (0 consecutive failures): Network error: Failed to ping master at 127.22.125.254:45811: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
I20260928 16:01:25.053342 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 7293
I20260928 16:01:25.053503 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.053104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:34389)
0928 16:01:25.053276 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.053371 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 16:01:25.054126 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:39101
--local_ip_for_outbound_sockets=127.22.125.194
--tserver_master_addrs=127.22.125.254:45811
--webserver_port=35113
--webserver_interface=127.22.125.194
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:01:25.054432 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.054003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:38907)
0928 16:01:25.054164 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.054257 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:01:25.054711 7465 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 16:01:25.378566 7468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:25.378815 7468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:25.378908 7468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:25.386289 7468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:25.386423 7468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:25.386615 7468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:01:25.396600 7468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:39101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=35113
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:25.397887 7468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:25.399209 7468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:25.412626 7476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:25.412742 7473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:25.412711 7474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:25.413470 7468 server_base.cc:1034] running on GCE node
I20260928 16:01:25.414124 7468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:25.415063 7468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:25.416404 7468 hybrid_clock.cc:648] HybridClock initialized: now 1790611285416363 us; error 52 us; skew 500 ppm
I20260928 16:01:25.416756 7468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:25 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611285, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:25.419759 7468 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:01:25.421258 7468 webserver.cc:460] Webserver started at http://127.22.125.194:35113/ using document root <none> and password file <none>
I20260928 16:01:25.421942 7468 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:25.422091 7468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:25.427273 7468 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260928 16:01:25.429886 7483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:25.431152 7468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:01:25.431340 7468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "76093d4c30d44531bfaa251d001c881f"
format_stamp: "Formatted at 2026-09-28 16:00:39 on dist-test-slave-9cqd"
server_key: "94f205a2e968bc1374dba28e98b67c5d"
server_key_iv: "52a1b6814657fda5b69acf17251d0e97"
server_key_version: "encryptionkey@0"
I20260928 16:01:25.432359 7468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:25.448781 7468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:25.452546 7468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:25.452811 7468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:25.453738 7468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:25.454967 7468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:25.455091 7468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:25.455226 7468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:25.455319 7468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:25.488888 7468 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:39101
I20260928 16:01:25.488988 7595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:39101 every 8 connection(s)
I20260928 16:01:25.490362 7468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:01:25.495064 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.493302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:38605)
0928 16:01:25.494002 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.494232 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":263,"threads_started":1}
W20260928 16:01:25.495716 7596 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:45811 (0 consecutive failures): Network error: Failed to ping master at 127.22.125.254:45811: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
I20260928 16:01:25.496551 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.496246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:38341)
0928 16:01:25.496387 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.496454 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 16:01:25.497316 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.497027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:49517)
0928 16:01:25.497141 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.497210 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 16:01:25.497596 7596 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 16:01:25.497587 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 7468
I20260928 16:01:25.498203 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:46851
--local_ip_for_outbound_sockets=127.22.125.195
--tserver_master_addrs=127.22.125.254:45811
--webserver_port=40577
--webserver_interface=127.22.125.195
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:25.826247 7599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:25.826491 7599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:25.826707 7599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:25.833573 7599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:25.833696 7599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:25.833868 7599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:01:25.843302 7599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:46851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=40577
--tserver_master_addrs=127.22.125.254:45811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:25.844666 7599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:25.846030 7599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:25.858827 7605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:25.858920 7604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:25.859161 7607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:25.859867 7599 server_base.cc:1034] running on GCE node
I20260928 16:01:25.860651 7599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:25.861793 7599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:25.862995 7599 hybrid_clock.cc:648] HybridClock initialized: now 1790611285862946 us; error 41 us; skew 500 ppm
I20260928 16:01:25.863270 7599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:25 dist-test-slave-9cqd krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611285, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:25.866480 7599 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:01:25.867911 7599 webserver.cc:460] Webserver started at http://127.22.125.195:40577/ using document root <none> and password file <none>
I20260928 16:01:25.868610 7599 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:25.868777 7599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:25.873919 7599 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 16:01:25.876554 7614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:25.877772 7599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:25.877944 7599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "9affaea8f0094261be224a439969eca0"
format_stamp: "Formatted at 2026-09-28 16:00:40 on dist-test-slave-9cqd"
server_key: "9b9fd5f5ed5cefb7950e7907c0aad765"
server_key_iv: "111326e18386bcd8ad90c8b1d4eb19b1"
server_key_version: "encryptionkey@0"
I20260928 16:01:25.878893 7599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:25.891765 7599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:25.895819 7599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:25.896149 7599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:25.897032 7599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:25.898267 7599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:25.898396 7599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:25.898519 7599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:25.898607 7599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:25.931665 7599 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:46851
I20260928 16:01:25.931753 7726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:46851 every 8 connection(s)
I20260928 16:01:25.933151 7599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:01:25.938073 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.936210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:36295)
0928 16:01:25.937004 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.937250 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":291,"threads_started":1}
W20260928 16:01:25.938730 7727 heartbeater.cc:644] Failed to heartbeat to 127.22.125.254:45811 (0 consecutive failures): Network error: Failed to ping master at 127.22.125.254:45811: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
I20260928 16:01:25.939451 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.939174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:47249)
0928 16:01:25.939294 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.939361 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260928 16:01:25.940227 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:25.939908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:52493)
0928 16:01:25.940014 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:25.940125 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 16:01:25.940438 7727 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 16:01:25.941553 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 7599
I20260928 16:01:26.055975 7731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:26.055251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:42199)
0928 16:01:26.055689 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:26.055768 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":231,"threads_started":1}
I20260928 16:01:26.462941 7732 raft_consensus.cc:482] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:26.463269 7732 raft_consensus.cc:504] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:26.465077 7732 leader_election.cc:288] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851), 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101)
Sep 28 16:01:26 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
Sep 28 16:01:26 dist-test-slave-9cqd krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 16:01:26.492235 7731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:26.465404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:46851 (local address 127.22.125.193:48863)
0928 16:01:26.465704 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:26.465756 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:26.466462 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:26.471345 (+ 4883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:26.471354 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:26.472428 (+ 1074us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:26.473634 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:26.473662 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.475921 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.475925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:26.478526 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:26.478542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.479937 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.479942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:26.480323 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:26.481308 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:26.481326 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:26.484655 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:26.488492 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:26.488496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:26.488500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:26.488989 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:26.489472 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:26.489479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:26.489484 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:26.489621 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:26.491029 (+ 1408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:26.491034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:26.491187 (+ 153us) client_negotiation.cc:769] Sending connection context
0928 16:01:26.491344 (+ 157us) client_negotiation.cc:240] Negotiation successful
0928 16:01:26.491772 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":498,"spinlock_wait_cycles":5888}
I20260928 16:01:26.492449 7734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:26.465535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:48863 (local address 127.22.125.195:46851)
0928 16:01:26.467694 (+ 2159us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:26.467714 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:26.467760 (+ 46us) server_negotiation.cc:407] Connection header received
0928 16:01:26.470222 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:26.470234 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:26.470732 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:26.471093 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:26.473848 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.475708 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.478675 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.479691 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.479889 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:26.484899 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:26.484906 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:26.484918 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:26.484953 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:26.488359 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:26.489140 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:26.489149 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:26.489156 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:26.489356 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:26.490023 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:26.490031 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:26.490038 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:26.490899 (+ 861us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:26.491090 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:26.491586 (+ 496us) server_negotiation.cc:299] Negotiation successful
0928 16:01:26.491993 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1938,"thread_start_us":1807,"threads_started":1}
I20260928 16:01:26.492774 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:26.465474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:39101 (local address 127.22.125.193:44631)
0928 16:01:26.468106 (+ 2632us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:26.468130 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:26.468284 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:26.471487 (+ 3203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:26.471491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:26.472428 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:26.473634 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:26.473662 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.475918 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.475924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:26.478527 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:26.478542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.479925 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.479932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:26.480328 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:26.481277 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:26.481316 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:26.484332 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:26.487777 (+ 3445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:26.487786 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:26.487800 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:26.488540 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:26.488985 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:26.488990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:26.488994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:26.489248 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:26.490388 (+ 1140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:26.490395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:26.490780 (+ 385us) client_negotiation.cc:769] Sending connection context
0928 16:01:26.491107 (+ 327us) client_negotiation.cc:240] Negotiation successful
0928 16:01:26.491579 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2445,"mutex_wait_us":554,"spinlock_wait_cycles":12288,"thread_start_us":2235,"threads_started":1}
I20260928 16:01:26.492901 7735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:26.465843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:44631 (local address 127.22.125.194:39101)
0928 16:01:26.468682 (+ 2839us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:26.468695 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:26.468730 (+ 35us) server_negotiation.cc:407] Connection header received
0928 16:01:26.470766 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:26.470776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:26.471191 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:26.471524 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:26.473848 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.475708 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.478675 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:26.479690 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:26.479889 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:26.484659 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:26.484670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:26.484689 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:26.484749 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:26.487634 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:26.488692 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:26.488698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:26.488703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:26.488865 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:26.489393 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:26.489398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:26.489403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:26.490133 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:26.490316 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:26.492103 (+ 1787us) server_negotiation.cc:299] Negotiation successful
0928 16:01:26.492515 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2680,"thread_start_us":2546,"threads_started":1}
W20260928 16:01:26.496346 7353 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
W20260928 16:01:26.497256 7350 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:26.497429 7350 leader_election.cc:302] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26a46693e0f9430b85f4df38e8375138; no voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0
I20260928 16:01:26.497898 7732 raft_consensus.cc:2726] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:01:26.498903 7736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:26.498131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:49243)
0928 16:01:26.498555 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:26.498686 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":214,"threads_started":1}
I20260928 16:01:26.941686 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:26.940901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:37857)
0928 16:01:26.941327 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:26.941461 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":197,"threads_started":1}
I20260928 16:01:27.057518 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:27.056882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:43867)
0928 16:01:27.057277 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:27.057360 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":196,"threads_started":1}
I20260928 16:01:27.500527 7739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:27.499872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:42983)
0928 16:01:27.500301 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:27.500380 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":241,"threads_started":1}
I20260928 16:01:27.943171 7740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:27.942522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:59223)
0928 16:01:27.942937 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:27.943012 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":234,"threads_started":1}
I20260928 16:01:28.059121 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:28.058468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:57595)
0928 16:01:28.058883 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:28.058959 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":206,"threads_started":1}
I20260928 16:01:28.288733 7742 raft_consensus.cc:482] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:28.288923 7742 raft_consensus.cc:504] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:28.289556 7742 leader_election.cc:288] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851), 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101)
W20260928 16:01:28.290539 7353 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
W20260928 16:01:28.290709 7350 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:28.290867 7350 leader_election.cc:302] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26a46693e0f9430b85f4df38e8375138; no voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0
I20260928 16:01:28.291178 7742 raft_consensus.cc:2726] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:01:28.501978 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:28.501340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:36583)
0928 16:01:28.501735 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:28.501810 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":204,"threads_started":1}
I20260928 16:01:28.944870 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:28.944174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:38041)
0928 16:01:28.944620 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:28.944698 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":216,"threads_started":1}
I20260928 16:01:29.060572 7745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:29.059884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:34295)
0928 16:01:29.060330 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:29.060404 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":256,"threads_started":1}
I20260928 16:01:29.503631 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:29.502887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:55025)
0928 16:01:29.503336 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:29.503421 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":217,"threads_started":1}
I20260928 16:01:29.946455 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:29.945812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:57283)
0928 16:01:29.946217 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:29.946290 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":203,"threads_started":1}
I20260928 16:01:30.062224 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:30.061460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:55171)
0928 16:01:30.061933 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:30.062009 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":217,"threads_started":1}
I20260928 16:01:30.499204 7749 raft_consensus.cc:482] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:30.499466 7749 raft_consensus.cc:504] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:30.500226 7749 leader_election.cc:288] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851), 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101)
W20260928 16:01:30.501152 7353 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
W20260928 16:01:30.501360 7350 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:30.501493 7350 leader_election.cc:302] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26a46693e0f9430b85f4df38e8375138; no voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0
I20260928 16:01:30.501793 7749 raft_consensus.cc:2726] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:01:30.505079 7750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:30.504440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:55917)
0928 16:01:30.504836 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:30.504927 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":202,"threads_started":1}
I20260928 16:01:30.948186 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:30.947431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:52529)
0928 16:01:30.947855 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:30.947950 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":220,"threads_started":1}
I20260928 16:01:31.063948 7752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:31.063263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:50119)
0928 16:01:31.063697 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:31.063796 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":213,"threads_started":1}
I20260928 16:01:31.506875 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:31.506134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:37213)
0928 16:01:31.506606 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:31.506696 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":250,"threads_started":1}
I20260928 16:01:31.949769 7754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:31.949016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:51361)
0928 16:01:31.949510 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:31.949592 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":265,"threads_started":1}
I20260928 16:01:32.065594 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:32.064895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:49815)
0928 16:01:32.065328 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:32.065402 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":261,"threads_started":1}
I20260928 16:01:32.508704 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:32.507868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:32955)
0928 16:01:32.508442 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:32.508527 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":282,"threads_started":1}
I20260928 16:01:32.663203 7757 raft_consensus.cc:482] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:32.663430 7757 raft_consensus.cc:504] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:32.664220 7757 leader_election.cc:288] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851), 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101)
W20260928 16:01:32.665148 7353 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
W20260928 16:01:32.665342 7350 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:32.665494 7350 leader_election.cc:302] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26a46693e0f9430b85f4df38e8375138; no voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0
I20260928 16:01:32.665747 7757 raft_consensus.cc:2726] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:01:32.951452 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:32.950661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:35239)
0928 16:01:32.951192 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:32.951273 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":264,"threads_started":1}
I20260928 16:01:33.067157 7759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:33.066401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:43781)
0928 16:01:33.066866 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:33.066981 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":226,"threads_started":1}
I20260928 16:01:33.510402 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:33.509711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:46135)
0928 16:01:33.510144 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:33.510223 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":233,"threads_started":1}
I20260928 16:01:33.953231 7761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:33.952525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:42731)
0928 16:01:33.952939 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:33.953022 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":211,"threads_started":1}
I20260928 16:01:34.068838 7762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:34.068008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:46673)
0928 16:01:34.068541 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:34.068647 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":216,"threads_started":1}
I20260928 16:01:34.512140 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:34.511252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:53563)
0928 16:01:34.511814 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:34.511925 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":334,"threads_started":1}
I20260928 16:01:34.954811 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:34.954138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:33711)
0928 16:01:34.954575 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:34.954650 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":224,"threads_started":1}
I20260928 16:01:35.070565 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:35.069830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:52175)
0928 16:01:35.070335 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:35.070410 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":229,"threads_started":1}
I20260928 16:01:35.513846 7766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:35.513061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:41837)
0928 16:01:35.513514 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:35.513620 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":224,"threads_started":1}
I20260928 16:01:35.956517 7767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:35.955802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:41849)
0928 16:01:35.956267 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:35.956342 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":239,"threads_started":1}
I20260928 16:01:36.072147 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:36.071365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:41383)
0928 16:01:36.071784 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:36.071873 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":227,"threads_started":1}
I20260928 16:01:36.515609 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:36.514809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:53299)
0928 16:01:36.515354 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:36.515441 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":291,"threads_started":1}
I20260928 16:01:36.958051 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:36.957351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:36947)
0928 16:01:36.957795 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:36.957890 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260928 16:01:37.073637 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:37.072943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:46103)
0928 16:01:37.073401 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:37.073479 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":216,"threads_started":1}
I20260928 16:01:37.517299 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:37.516556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:43721)
0928 16:01:37.517016 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:37.517095 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":228,"threads_started":1}
I20260928 16:01:37.959709 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:37.959019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:56181)
0928 16:01:37.959459 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:37.959538 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":231,"threads_started":1}
I20260928 16:01:38.075188 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:38.074486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:47683)
0928 16:01:38.074924 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:38.075022 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":221,"threads_started":1}
I20260928 16:01:38.519148 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:38.518445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:44041)
0928 16:01:38.518884 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:38.518981 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":232,"threads_started":1}
I20260928 16:01:38.961520 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:38.960667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:43885)
0928 16:01:38.961159 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:38.961296 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":255,"threads_started":1}
I20260928 16:01:39.076709 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:39.075999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:47603)
0928 16:01:39.076461 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:39.076539 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":264,"threads_started":1}
I20260928 16:01:39.520766 7778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:39.520006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:59969)
0928 16:01:39.520482 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:39.520569 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
I20260928 16:01:39.963366 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:39.962363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:43731)
0928 16:01:39.962980 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:39.963142 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":313,"threads_started":1}
I20260928 16:01:40.078517 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:40.077792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.193:51247)
0928 16:01:40.078235 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:40.078335 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":221,"threads_started":1}
I20260928 16:01:40.164752 7781 raft_consensus.cc:482] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:40.165011 7781 raft_consensus.cc:504] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9affaea8f0094261be224a439969eca0" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 46851 } } peers { permanent_uuid: "76093d4c30d44531bfaa251d001c881f" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 39101 } } peers { permanent_uuid: "26a46693e0f9430b85f4df38e8375138" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 38495 } }
I20260928 16:01:40.165758 7781 leader_election.cc:288] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851), 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101)
W20260928 16:01:40.166652 7353 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9affaea8f0094261be224a439969eca0 (127.22.125.195:46851): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
W20260928 16:01:40.166846 7350 leader_election.cc:341] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 76093d4c30d44531bfaa251d001c881f (127.22.125.194:39101): Not found: Tablet not found: 312171ef81c141b0b767a1b9f3dfd4f0
I20260928 16:01:40.166958 7350 leader_election.cc:302] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26a46693e0f9430b85f4df38e8375138; no voters: 76093d4c30d44531bfaa251d001c881f, 9affaea8f0094261be224a439969eca0
I20260928 16:01:40.167162 7781 raft_consensus.cc:2726] T 312171ef81c141b0b767a1b9f3dfd4f0 P 26a46693e0f9430b85f4df38e8375138 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 16:01:40.524286 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:40.521935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.194:36659)
0928 16:01:40.523803 (+ 1868us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:40.524106 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1733,"thread_start_us":1273,"threads_started":1}
I20260928 16:01:40.947808 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 7293
I20260928 16:01:40.965343 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:40.964455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45811 (local address 127.22.125.195:43413)
0928 16:01:40.964962 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:40.965082 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.254:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":230,"threads_started":1}
I20260928 16:01:40.971367 7783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:40.948891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.22.125.193:38495)
0928 16:01:40.949455 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:40.949469 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:40.949630 (+ 161us) server_negotiation.cc:407] Connection header received
0928 16:01:40.949866 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:40.949877 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:40.950201 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:40.950395 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:40.951834 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:40.952827 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:40.954092 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:40.954621 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:40.954720 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:40.966121 (+ 11401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:40.966133 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:40.966147 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:40.966179 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:40.968906 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:40.969537 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:40.969543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:40.969546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:40.969669 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:40.970003 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:40.970007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:40.970008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:40.970347 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:40.970634 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:40.970896 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 16:01:40.971107 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":223,"threads_started":1}
I20260928 16:01:40.972685 7460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:41.061842 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 7293
I20260928 16:01:41.079254 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 7468
I20260928 16:01:41.092705 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:41.080025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.22.125.194:39101)
0928 16:01:41.080541 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:41.080546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:41.080562 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:01:41.080619 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:41.080622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:41.080682 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:41.080773 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:41.082242 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:41.083226 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:41.084535 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:41.084967 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:41.085061 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:41.087545 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:41.087550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:41.087553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:41.087585 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:41.090321 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:41.090886 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:41.090890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:41.090892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:41.091037 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:41.091354 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:41.091357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:41.091359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:41.091610 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:41.091808 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:41.092129 (+ 321us) server_negotiation.cc:299] Negotiation successful
0928 16:01:41.092297 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":285,"threads_started":1}
I20260928 16:01:41.093360 7591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:41.176148 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 7468
I20260928 16:01:41.193805 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 7599
I20260928 16:01:41.206954 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:41.194631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.22.125.195:46851)
0928 16:01:41.195147 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:41.195152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:41.195170 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:01:41.195228 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:41.195231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:41.195300 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:41.195395 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:41.196820 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:41.197621 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:41.198765 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:41.199230 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:41.199326 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:41.201905 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:41.201909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:41.201912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:41.201942 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:41.204508 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:41.205101 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:41.205107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:41.205109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:41.205244 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:41.205605 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:41.205610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:41.205612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:41.205880 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:41.206108 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:41.206325 (+ 217us) server_negotiation.cc:299] Negotiation successful
0928 16:01:41.206481 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":265,"threads_started":1}
I20260928 16:01:41.207687 7722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:41.290835 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 7599
2026-09-28T16:01:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64406 ms)
[----------] 30 tests from SecurityITest (295284 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:01:41 dist-test-slave-9cqd krb5kdc[7792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:01:41 dist-test-slave-9cqd krb5kdc[7792](info): set up 2 sockets
Sep 28 16:01:41 dist-test-slave-9cqd krb5kdc[7792](info): commencing operation
krb5kdc: starting...
W20260928 16:01:42.406088 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:01:42 dist-test-slave-9cqd krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:01:42.639454 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:40073
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:40073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:42.963528 7805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:42.963853 7805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:42.964005 7805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:42.971338 7805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:01:42.971468 7805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:42.971571 7805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:42.971673 7805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:01:42.971736 7805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:01:42.981398 7805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:40073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:42.982641 7805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:42.983866 7805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:42.995904 7810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:42.995901 7813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:42.995946 7811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:42.996393 7805 server_base.cc:1034] running on GCE node
I20260928 16:01:42.997301 7805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:42.998651 7805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:42.999888 7805 hybrid_clock.cc:648] HybridClock initialized: now 1790611302999846 us; error 40 us; skew 500 ppm
I20260928 16:01:43.000284 7805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:42 dist-test-slave-9cqd krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611302, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:43.003232 7805 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:01:43.004771 7805 webserver.cc:460] Webserver started at http://127.22.125.254:38513/ using document root <none> and password file <none>
I20260928 16:01:43.005437 7805 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:43.005587 7805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:43.005888 7805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:43.008857 7805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "70ceeaadc7154e43aade849152ed473b"
format_stamp: "Formatted at 2026-09-28 16:01:43 on dist-test-slave-9cqd"
server_key: "e10853217acbf77e1918862883f2e808"
server_key_iv: "785e3be7f98366f724f9673cc3892847"
server_key_version: "encryptionkey@0"
I20260928 16:01:43.009586 7805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "70ceeaadc7154e43aade849152ed473b"
format_stamp: "Formatted at 2026-09-28 16:01:43 on dist-test-slave-9cqd"
server_key: "e10853217acbf77e1918862883f2e808"
server_key_iv: "785e3be7f98366f724f9673cc3892847"
server_key_version: "encryptionkey@0"
I20260928 16:01:43.014385 7805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:01:43.017598 7820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:43.018770 7805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 16:01:43.018949 7805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "70ceeaadc7154e43aade849152ed473b"
format_stamp: "Formatted at 2026-09-28 16:01:43 on dist-test-slave-9cqd"
server_key: "e10853217acbf77e1918862883f2e808"
server_key_iv: "785e3be7f98366f724f9673cc3892847"
server_key_version: "encryptionkey@0"
I20260928 16:01:43.019129 7805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:43.038882 7805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:43.042853 7805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:43.043129 7805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:43.061434 7805 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:40073
I20260928 16:01:43.061457 7871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:40073 every 8 connection(s)
I20260928 16:01:43.062788 7805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:01:43.066289 7872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:43.070565 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 7805
I20260928 16:01:43.070788 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:01:43.071646 23031 external_mini_cluster.cc:1428] Setting key cb22790b50e1dd543332ac02a9d8c222
I20260928 16:01:43.077875 7872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b: Bootstrap starting.
I20260928 16:01:43.081367 7872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:43.082582 7872 log.cc:824] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:43.085999 7872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b: No bootstrap required, opened a new log
Sep 28 16:01:43 dist-test-slave-9cqd krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:43.093688 7872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ceeaadc7154e43aade849152ed473b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40073 } }
I20260928 16:01:43.094055 7872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:43.094175 7872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70ceeaadc7154e43aade849152ed473b, State: Initialized, Role: FOLLOWER
I20260928 16:01:43.094841 7872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ceeaadc7154e43aade849152ed473b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40073 } }
I20260928 16:01:43.095109 7872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:01:43.095250 7872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:01:43.095443 7872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:43.095991 7875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:43.073866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.22.125.254:40073)
0928 16:01:43.074711 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:43.074723 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:43.074768 (+ 45us) server_negotiation.cc:407] Connection header received
0928 16:01:43.076584 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:43.076606 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:43.076948 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:43.077255 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:43.079029 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:43.080475 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:43.082481 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:43.083552 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:43.083777 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:43.088455 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:43.088463 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:43.088480 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:43.088520 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:43.091631 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:43.092386 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:43.092392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:43.092399 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:43.092579 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:43.092983 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:43.092990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:43.092997 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:43.093521 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:43.093973 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:43.094431 (+ 458us) server_negotiation.cc:299] Negotiation successful
0928 16:01:43.094844 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":395,"threads_started":1}
I20260928 16:01:43.098274 7872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ceeaadc7154e43aade849152ed473b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40073 } }
I20260928 16:01:43.098837 7872 leader_election.cc:302] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70ceeaadc7154e43aade849152ed473b; no voters:
I20260928 16:01:43.099421 7872 leader_election.cc:288] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:01:43.099562 7877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:43.100394 7877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [term 1 LEADER]: Becoming Leader. State: Replica: 70ceeaadc7154e43aade849152ed473b, State: Running, Role: LEADER
I20260928 16:01:43.100936 7877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ceeaadc7154e43aade849152ed473b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40073 } }
I20260928 16:01:43.101598 7872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:01:43.103475 7878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70ceeaadc7154e43aade849152ed473b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ceeaadc7154e43aade849152ed473b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40073 } } }
I20260928 16:01:43.103840 7878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:43.105041 7879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70ceeaadc7154e43aade849152ed473b. Latest consensus state: current_term: 1 leader_uuid: "70ceeaadc7154e43aade849152ed473b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ceeaadc7154e43aade849152ed473b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 40073 } } }
I20260928 16:01:43.105430 7879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:43.112295 7882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:01:43.116951 7882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:01:43.125288 7882 catalog_manager.cc:1347] Generated new cluster ID: df5a07ed1ff5421f998f761b3950a1a8
I20260928 16:01:43.125393 7882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:01:43.136268 7882 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:01:43.138039 7882 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:01:43.144155 7882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b: Generated new TSK 0
I20260928 16:01:43.145045 7882 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 16:01:43.207453 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40073
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:43.532341 7898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:43.532588 7898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:43.532745 7898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:43.539722 7898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:43.539813 7898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:43.539978 7898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:01:43.549417 7898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:43.550581 7898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:43.551860 7898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:43.565178 7906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:43.565207 7903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:43.565122 7904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:43.566632 7898 server_base.cc:1034] running on GCE node
I20260928 16:01:43.567318 7898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:43.568516 7898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:43.571926 7898 hybrid_clock.cc:648] HybridClock initialized: now 1790611303571850 us; error 43 us; skew 500 ppm
I20260928 16:01:43.572322 7898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:43 dist-test-slave-9cqd krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611303, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:43.576706 7898 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:01:43.578150 7898 webserver.cc:460] Webserver started at http://127.22.125.193:45377/ using document root <none> and password file <none>
I20260928 16:01:43.579031 7898 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:43.579200 7898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:43.579504 7898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:43.582549 7898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "d6dc38b0fb02468db92da7cae2f63ce1"
format_stamp: "Formatted at 2026-09-28 16:01:43 on dist-test-slave-9cqd"
server_key: "1c9540a74a004644566d471c5b3196ae"
server_key_iv: "e6deea7fb5bfdcf44b2ce24e75e388c4"
server_key_version: "encryptionkey@0"
I20260928 16:01:43.583560 7898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "d6dc38b0fb02468db92da7cae2f63ce1"
format_stamp: "Formatted at 2026-09-28 16:01:43 on dist-test-slave-9cqd"
server_key: "1c9540a74a004644566d471c5b3196ae"
server_key_iv: "e6deea7fb5bfdcf44b2ce24e75e388c4"
server_key_version: "encryptionkey@0"
I20260928 16:01:43.590221 7898 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.001s
I20260928 16:01:43.593926 7913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:43.595546 7898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:01:43.595745 7898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "d6dc38b0fb02468db92da7cae2f63ce1"
format_stamp: "Formatted at 2026-09-28 16:01:43 on dist-test-slave-9cqd"
server_key: "1c9540a74a004644566d471c5b3196ae"
server_key_iv: "e6deea7fb5bfdcf44b2ce24e75e388c4"
server_key_version: "encryptionkey@0"
I20260928 16:01:43.595969 7898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:43.618762 7898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:43.622522 7898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:43.622763 7898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:43.623610 7898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:43.624887 7898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:43.624956 7898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:43.625039 7898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:43.625082 7898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:43.656831 7898 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:39425
I20260928 16:01:43.656912 8026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:39425 every 8 connection(s)
I20260928 16:01:43.658064 7898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:01:43.659821 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 7898
I20260928 16:01:43.660061 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:01:43.660916 23031 external_mini_cluster.cc:1428] Setting key 36bf6a8d602a6c6e7c476d36711bbc84
Sep 28 16:01:43 dist-test-slave-9cqd krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611303, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:43.678819 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:43.660560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:42169 (local address 127.22.125.254:40073)
0928 16:01:43.661106 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:43.661112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:43.662138 (+ 1026us) server_negotiation.cc:407] Connection header received
0928 16:01:43.663960 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:43.663969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:43.664081 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:43.664200 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:43.665615 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:43.666752 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:43.668341 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:43.669062 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:43.669206 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:43.673052 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:43.673057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:43.673060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:43.673089 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:43.675634 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:43.676612 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:43.676617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:43.676619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:43.676738 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:43.677210 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:43.677213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:43.677215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:43.677456 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:43.677562 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:43.678408 (+ 846us) server_negotiation.cc:299] Negotiation successful
0928 16:01:43.678566 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":265,"threads_started":1}
I20260928 16:01:43.680284 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:43.661115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:40073 (local address 127.22.125.193:42169)
0928 16:01:43.661951 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:43.662000 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:43.663692 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:43.664400 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:43.664407 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:43.664903 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:43.665442 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:43.665459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:43.666888 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:43.666891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:43.668208 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:43.668221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:43.669282 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:43.669286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:43.669468 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:43.670127 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:43.670153 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:43.672836 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:43.675776 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:43.675788 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:43.675804 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:43.676467 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:43.676850 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:43.676857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:43.676864 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:43.677096 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:43.677588 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:43.677599 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:43.678015 (+ 416us) client_negotiation.cc:769] Sending connection context
0928 16:01:43.678318 (+ 303us) client_negotiation.cc:240] Negotiation successful
0928 16:01:43.678877 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":352,"threads_started":1}
I20260928 16:01:43.681844 8027 heartbeater.cc:347] Connected to a master server at 127.22.125.254:40073
I20260928 16:01:43.682147 8027 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:43.682937 8027 heartbeater.cc:511] Master 127.22.125.254:40073 requested a full tablet report, sending...
I20260928 16:01:43.684974 7837 ts_manager.cc:194] Registered new tserver with Master: d6dc38b0fb02468db92da7cae2f63ce1 (127.22.125.193:39425)
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 16:01:43.686421 7837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:42169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 16:01:43.716938 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40073
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:44.037294 8033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:44.037547 8033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:44.037639 8033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:44.044734 8033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:44.044835 8033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:44.044950 8033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:01:44.054394 8033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:44.055511 8033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:44.056798 8033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:44.069928 8041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:44.069971 8039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:44.070627 8038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:44.070904 8033 server_base.cc:1034] running on GCE node
I20260928 16:01:44.071534 8033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:44.072477 8033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:44.073705 8033 hybrid_clock.cc:648] HybridClock initialized: now 1790611304073654 us; error 43 us; skew 500 ppm
I20260928 16:01:44.074003 8033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:44 dist-test-slave-9cqd krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611304, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:44.077224 8033 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:01:44.078711 8033 webserver.cc:460] Webserver started at http://127.22.125.194:32907/ using document root <none> and password file <none>
I20260928 16:01:44.079387 8033 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:44.079587 8033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:44.079959 8033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:44.082789 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "0d0914db9cd1404ca86a8404d1916c70"
format_stamp: "Formatted at 2026-09-28 16:01:44 on dist-test-slave-9cqd"
server_key: "3a2d89de7bdb116b14bb901479f5c610"
server_key_iv: "d1f427d71662d194709231632e2fc3fd"
server_key_version: "encryptionkey@0"
I20260928 16:01:44.083541 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "0d0914db9cd1404ca86a8404d1916c70"
format_stamp: "Formatted at 2026-09-28 16:01:44 on dist-test-slave-9cqd"
server_key: "3a2d89de7bdb116b14bb901479f5c610"
server_key_iv: "d1f427d71662d194709231632e2fc3fd"
server_key_version: "encryptionkey@0"
I20260928 16:01:44.088562 8033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260928 16:01:44.091849 8048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:44.093281 8033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:01:44.093446 8033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "0d0914db9cd1404ca86a8404d1916c70"
format_stamp: "Formatted at 2026-09-28 16:01:44 on dist-test-slave-9cqd"
server_key: "3a2d89de7bdb116b14bb901479f5c610"
server_key_iv: "d1f427d71662d194709231632e2fc3fd"
server_key_version: "encryptionkey@0"
I20260928 16:01:44.093647 8033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:44.127221 8033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:44.131166 8033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:44.131461 8033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:44.132444 8033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:44.133741 8033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:44.133821 8033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:44.134006 8033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:44.134057 8033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:44.166963 8033 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:34717
I20260928 16:01:44.167004 8160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:34717 every 8 connection(s)
I20260928 16:01:44.168300 8033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:01:44.178215 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8033
I20260928 16:01:44.178448 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:01:44.178948 23031 external_mini_cluster.cc:1428] Setting key 1007a3f451f13b413e91ba3e53dfec3a
Sep 28 16:01:44 dist-test-slave-9cqd krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611304, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:44.188796 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.170799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:58547 (local address 127.22.125.254:40073)
0928 16:01:44.171007 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.171014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.172309 (+ 1295us) server_negotiation.cc:407] Connection header received
0928 16:01:44.174036 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.174041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.174107 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.174194 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:44.175588 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.176372 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.177818 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.178251 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.178391 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.181929 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:44.181938 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:44.181946 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:44.181988 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:44.185614 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.186294 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.186308 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.186314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.186472 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.186860 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.186868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.186874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.187268 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:44.187441 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.188084 (+ 643us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.188348 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:01:44.189431 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.171358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:40073 (local address 127.22.125.194:58547)
0928 16:01:44.172146 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:44.172192 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:44.173784 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:44.174389 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:44.174397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:44.174899 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:44.175436 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.175452 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.176503 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.176507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.177698 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.177710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.178478 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.178486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.178703 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.179414 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:44.179439 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:44.181709 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:44.185741 (+ 4032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:44.185747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:44.185758 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:44.186176 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:44.186580 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:44.186582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:44.186585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:44.186749 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:44.187397 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:44.187402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:44.187705 (+ 303us) client_negotiation.cc:769] Sending connection context
0928 16:01:44.187923 (+ 218us) client_negotiation.cc:240] Negotiation successful
0928 16:01:44.188338 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":285,"threads_started":1}
I20260928 16:01:44.191045 8161 heartbeater.cc:347] Connected to a master server at 127.22.125.254:40073
I20260928 16:01:44.191352 8161 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:44.192095 8161 heartbeater.cc:511] Master 127.22.125.254:40073 requested a full tablet report, sending...
I20260928 16:01:44.193547 7837 ts_manager.cc:194] Registered new tserver with Master: 0d0914db9cd1404ca86a8404d1916c70 (127.22.125.194:34717)
I20260928 16:01:44.194330 7837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:58547
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 16:01:44.233355 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40073
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:44.555660 8166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:44.555994 8166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:44.556180 8166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:44.563035 8166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:44.563171 8166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:44.563364 8166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:01:44.574021 8166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:40073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:44.575259 8166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:44.576613 8166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:44.589432 8171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:44.589474 8172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:44.589382 8174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:44.590279 8166 server_base.cc:1034] running on GCE node
I20260928 16:01:44.590837 8166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:44.591717 8166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:44.592945 8166 hybrid_clock.cc:648] HybridClock initialized: now 1790611304592907 us; error 52 us; skew 500 ppm
I20260928 16:01:44.593304 8166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:44 dist-test-slave-9cqd krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611304, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:44.596287 8166 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:01:44.597867 8166 webserver.cc:460] Webserver started at http://127.22.125.195:33227/ using document root <none> and password file <none>
I20260928 16:01:44.598568 8166 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:44.598711 8166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:44.599012 8166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:44.602028 8166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "5ccd7886e4454d9b996423e6e269beda"
format_stamp: "Formatted at 2026-09-28 16:01:44 on dist-test-slave-9cqd"
server_key: "0dc0ec5f5e86c0588ddde9d7e4e25ad4"
server_key_iv: "7ef07057677bb611c004d3ef824df941"
server_key_version: "encryptionkey@0"
I20260928 16:01:44.602770 8166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "5ccd7886e4454d9b996423e6e269beda"
format_stamp: "Formatted at 2026-09-28 16:01:44 on dist-test-slave-9cqd"
server_key: "0dc0ec5f5e86c0588ddde9d7e4e25ad4"
server_key_iv: "7ef07057677bb611c004d3ef824df941"
server_key_version: "encryptionkey@0"
I20260928 16:01:44.607695 8166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 16:01:44.610977 8181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:44.612244 8166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:01:44.612407 8166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "5ccd7886e4454d9b996423e6e269beda"
format_stamp: "Formatted at 2026-09-28 16:01:44 on dist-test-slave-9cqd"
server_key: "0dc0ec5f5e86c0588ddde9d7e4e25ad4"
server_key_iv: "7ef07057677bb611c004d3ef824df941"
server_key_version: "encryptionkey@0"
I20260928 16:01:44.612568 8166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:44.627297 8166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:44.630924 8166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:44.631192 8166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:44.632128 8166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:44.633349 8166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:44.633421 8166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:44.633602 8166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:44.633656 8166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:44.666728 8166 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:44947
I20260928 16:01:44.666786 8293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:44947 every 8 connection(s)
I20260928 16:01:44.667937 8166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:01:44.674669 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8166
I20260928 16:01:44.674854 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:01:44.675309 23031 external_mini_cluster.cc:1428] Setting key 27eac67574acea72a7f7c3fdcec870fe
Sep 28 16:01:44 dist-test-slave-9cqd krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611304, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:44.687212 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.670467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:53953 (local address 127.22.125.254:40073)
0928 16:01:44.670645 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.670650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.671706 (+ 1056us) server_negotiation.cc:407] Connection header received
0928 16:01:44.673532 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.673538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.673661 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.673776 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:44.675309 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.676214 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.677953 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.678393 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.678466 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.681885 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:44.681890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:44.681893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:44.681923 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:44.684293 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.684972 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.684978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.684981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.685113 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.685544 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.685549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.685554 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.685824 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:44.685983 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.686645 (+ 662us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.686873 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:01:44.688130 8296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.670875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:40073 (local address 127.22.125.195:53953)
0928 16:01:44.671549 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:44.671588 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:44.673269 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:44.674010 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:44.674018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:44.674492 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:44.675137 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.675156 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.676347 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.676358 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.677756 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.677768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.678728 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.678732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.678931 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.679534 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:44.679555 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:44.681679 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:44.684424 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:44.684432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:44.684446 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:44.684849 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:44.685227 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:44.685231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:44.685235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:44.685422 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:44.685951 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:44.685958 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:44.686289 (+ 331us) client_negotiation.cc:769] Sending connection context
0928 16:01:44.686556 (+ 267us) client_negotiation.cc:240] Negotiation successful
0928 16:01:44.687034 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":263,"threads_started":1}
I20260928 16:01:44.689426 8027 heartbeater.cc:503] Master 127.22.125.254:40073 was elected leader, sending a full tablet report...
I20260928 16:01:44.689630 8294 heartbeater.cc:347] Connected to a master server at 127.22.125.254:40073
I20260928 16:01:44.689971 8294 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:44.690778 8294 heartbeater.cc:511] Master 127.22.125.254:40073 requested a full tablet report, sending...
I20260928 16:01:44.692176 7837 ts_manager.cc:194] Registered new tserver with Master: 5ccd7886e4454d9b996423e6e269beda (127.22.125.195:44947)
I20260928 16:01:44.692956 7837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:53953
I20260928 16:01:44.702667 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:01:44.720023 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.707043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.22.125.254:40073)
0928 16:01:44.707199 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.707204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.707693 (+ 489us) server_negotiation.cc:407] Connection header received
0928 16:01:44.707889 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.707893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.707950 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.708048 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:44.709675 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.710460 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.711820 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.712303 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.712440 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.714666 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:44.714672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:44.714675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:44.714704 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:44.717377 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.718035 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.718047 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.718050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.718175 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.718566 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.718574 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.718577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.718896 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:44.719160 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.719495 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.719713 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:01:44.723731 7837 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:01:44.730520 7837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:01:44.756459 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.742949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.22.125.195:44947)
0928 16:01:44.745270 (+ 2321us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.745278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.745311 (+ 33us) server_negotiation.cc:407] Connection header received
0928 16:01:44.745422 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.745434 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.745725 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.746937 (+ 1212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:44.747270 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.749426 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.753909 (+ 4483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.755296 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.755496 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.755679 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.755936 (+ 257us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.756114 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2135,"thread_start_us":1911,"threads_started":1}
I20260928 16:01:44.756594 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.742323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44947 (local address 127.0.0.1:38364)
0928 16:01:44.744467 (+ 2144us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:44.744486 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:44.744597 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:44.746006 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:44.746016 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:44.746068 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:44.746672 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.746688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.749626 (+ 2938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.749635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.752918 (+ 3283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.752940 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.755496 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.755504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.755692 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.755751 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 16:01:44.755883 (+ 132us) client_negotiation.cc:240] Negotiation successful
0928 16:01:44.756097 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2010,"mutex_wait_us":4,"thread_start_us":873,"threads_started":1}
I20260928 16:01:44.757118 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.741883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:34717 (local address 127.0.0.1:57126)
0928 16:01:44.743342 (+ 1459us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:44.743378 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:44.743527 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:44.746092 (+ 2565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:44.746101 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:44.746140 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:44.746794 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.746807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.750897 (+ 4090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.750906 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.754443 (+ 3537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.754453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.756256 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.756263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.756447 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.756486 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 16:01:44.756563 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 16:01:44.756702 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"mutex_wait_us":107,"thread_start_us":2325,"threads_started":1}
I20260928 16:01:44.757565 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.742068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.22.125.194:34717)
0928 16:01:44.744174 (+ 2106us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.744187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.744220 (+ 33us) server_negotiation.cc:407] Connection header received
0928 16:01:44.744347 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.744358 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.744652 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.747061 (+ 2409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:44.747391 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.750702 (+ 3311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.754590 (+ 3888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.756000 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.756857 (+ 857us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.757022 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.757095 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.757259 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1881,"thread_start_us":1732,"threads_started":1}
I20260928 16:01:44.760773 8229 tablet_service.cc:1467] Processing CreateTablet for tablet bc3ea94a742048c4b07b8f52c4310ef2 (DEFAULT_TABLE table=test-table [id=7f39919cffff461d9cee2963a64d3e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:44.760880 8096 tablet_service.cc:1467] Processing CreateTablet for tablet bc3ea94a742048c4b07b8f52c4310ef2 (DEFAULT_TABLE table=test-table [id=7f39919cffff461d9cee2963a64d3e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:44.762080 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.745225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59910)
0928 16:01:44.753891 (+ 8666us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:44.753909 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:44.754026 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:44.754630 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:44.754634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:44.754663 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:44.755000 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.755011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.757004 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.757010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.759361 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.759376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.761228 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.761234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.761420 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.761451 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 16:01:44.761559 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 16:01:44.761760 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8527,"thread_start_us":8393,"threads_started":1}
I20260928 16:01:44.762290 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.745033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.22.125.193:39425)
0928 16:01:44.753623 (+ 8590us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.753633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.754026 (+ 393us) server_negotiation.cc:407] Connection header received
0928 16:01:44.754166 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.754174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.754428 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.754614 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:44.755169 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.756828 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.759545 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.761012 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.761211 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.761381 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.761771 (+ 390us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.761927 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8396,"thread_start_us":8227,"threads_started":1}
I20260928 16:01:44.762437 8096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc3ea94a742048c4b07b8f52c4310ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:44.762768 8229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc3ea94a742048c4b07b8f52c4310ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:44.765810 7962 tablet_service.cc:1467] Processing CreateTablet for tablet bc3ea94a742048c4b07b8f52c4310ef2 (DEFAULT_TABLE table=test-table [id=7f39919cffff461d9cee2963a64d3e14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:44.767143 7962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc3ea94a742048c4b07b8f52c4310ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:44.774003 8312 tablet_bootstrap.cc:492] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Bootstrap starting.
I20260928 16:01:44.777014 8313 tablet_bootstrap.cc:492] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Bootstrap starting.
I20260928 16:01:44.777575 8312 tablet_bootstrap.cc:654] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:44.778877 8312 log.cc:824] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:44.780769 8313 tablet_bootstrap.cc:654] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:44.781998 8313 log.cc:824] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:44.782269 8312 tablet_bootstrap.cc:492] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: No bootstrap required, opened a new log
I20260928 16:01:44.782378 8315 tablet_bootstrap.cc:492] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Bootstrap starting.
I20260928 16:01:44.782578 8312 ts_tablet_manager.cc:1397] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260928 16:01:44.785346 8313 tablet_bootstrap.cc:492] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: No bootstrap required, opened a new log
I20260928 16:01:44.785600 8313 ts_tablet_manager.cc:1397] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 16:01:44.786127 8315 tablet_bootstrap.cc:654] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:44.787396 8315 log.cc:824] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:44.790050 8315 tablet_bootstrap.cc:492] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: No bootstrap required, opened a new log
I20260928 16:01:44.790339 8315 ts_tablet_manager.cc:1397] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260928 16:01:44.793094 8312 raft_consensus.cc:348] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.793502 8312 raft_consensus.cc:374] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:44.793619 8312 raft_consensus.cc:729] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d0914db9cd1404ca86a8404d1916c70, State: Initialized, Role: FOLLOWER
I20260928 16:01:44.794194 8312 consensus_queue.cc:260] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.795325 8161 heartbeater.cc:503] Master 127.22.125.254:40073 was elected leader, sending a full tablet report...
I20260928 16:01:44.795981 8312 ts_tablet_manager.cc:1428] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Time spent starting tablet: real 0.013s user 0.002s sys 0.010s
I20260928 16:01:44.797196 8313 raft_consensus.cc:348] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.797662 8313 raft_consensus.cc:374] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:44.797782 8313 raft_consensus.cc:729] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccd7886e4454d9b996423e6e269beda, State: Initialized, Role: FOLLOWER
I20260928 16:01:44.798497 8313 consensus_queue.cc:260] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.799706 8294 heartbeater.cc:503] Master 127.22.125.254:40073 was elected leader, sending a full tablet report...
I20260928 16:01:44.800467 8313 ts_tablet_manager.cc:1428] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260928 16:01:44.801923 8315 raft_consensus.cc:348] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.802392 8315 raft_consensus.cc:374] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:44.802525 8315 raft_consensus.cc:729] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6dc38b0fb02468db92da7cae2f63ce1, State: Initialized, Role: FOLLOWER
I20260928 16:01:44.803189 8315 consensus_queue.cc:260] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.805034 8315 ts_tablet_manager.cc:1428] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260928 16:01:44.814719 8318 raft_consensus.cc:482] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:44.814987 8318 raft_consensus.cc:504] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.816632 8318 leader_election.cc:288] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6dc38b0fb02468db92da7cae2f63ce1 (127.22.125.193:39425), 5ccd7886e4454d9b996423e6e269beda (127.22.125.195:44947)
I20260928 16:01:44.823839 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.817016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:59953 (local address 127.22.125.195:44947)
0928 16:01:44.817218 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.817226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.818027 (+ 801us) server_negotiation.cc:407] Connection header received
0928 16:01:44.818220 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.818224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.818311 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.818400 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:44.819054 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.819749 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.821637 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.822729 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.822934 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.823009 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.823562 (+ 553us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.823657 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:01:44.824088 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.817059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:44947 (local address 127.22.125.194:59953)
0928 16:01:44.817919 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:44.817938 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:44.818088 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:44.818448 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:44.818451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:44.818521 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:44.818866 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.818875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.819918 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.819922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.821492 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.821504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.823122 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.823131 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.823354 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.823388 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 16:01:44.823509 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 16:01:44.823665 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":32,"spinlock_wait_cycles":16128,"thread_start_us":444,"threads_started":1}
I20260928 16:01:44.824160 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.817015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:48203 (local address 127.22.125.193:39425)
0928 16:01:44.817243 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.817252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.817922 (+ 670us) server_negotiation.cc:407] Connection header received
0928 16:01:44.818118 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.818126 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.818206 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.818285 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:44.818964 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.819711 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.821928 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.823120 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.823272 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.823326 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.823749 (+ 423us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.823897 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:01:44.824805 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.816764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.22.125.194:48203)
0928 16:01:44.817808 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:44.817833 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:44.817949 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:44.818343 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:44.818351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:44.818380 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:44.818811 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.818825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.819842 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.819850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.821778 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:44.821791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.823344 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.823349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:44.823544 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.823608 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 16:01:44.824306 (+ 698us) client_negotiation.cc:240] Negotiation successful
0928 16:01:44.824440 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":129,"thread_start_us":707,"threads_started":1}
I20260928 16:01:44.825039 8249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3ea94a742048c4b07b8f52c4310ef2" candidate_uuid: "0d0914db9cd1404ca86a8404d1916c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ccd7886e4454d9b996423e6e269beda" is_pre_election: true
I20260928 16:01:44.825578 8249 raft_consensus.cc:2454] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d0914db9cd1404ca86a8404d1916c70 in term 0.
I20260928 16:01:44.825723 7982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3ea94a742048c4b07b8f52c4310ef2" candidate_uuid: "0d0914db9cd1404ca86a8404d1916c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" is_pre_election: true
I20260928 16:01:44.826285 7982 raft_consensus.cc:2454] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d0914db9cd1404ca86a8404d1916c70 in term 0.
I20260928 16:01:44.826320 8049 leader_election.cc:302] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d0914db9cd1404ca86a8404d1916c70, 5ccd7886e4454d9b996423e6e269beda; no voters:
I20260928 16:01:44.826737 8318 raft_consensus.cc:2781] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:01:44.826912 8318 raft_consensus.cc:482] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:01:44.827028 8318 raft_consensus.cc:3037] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:44.829313 8318 raft_consensus.cc:504] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.829938 8318 leader_election.cc:288] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [CANDIDATE]: Term 1 election: Requested vote from peers d6dc38b0fb02468db92da7cae2f63ce1 (127.22.125.193:39425), 5ccd7886e4454d9b996423e6e269beda (127.22.125.195:44947)
I20260928 16:01:44.830282 7982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3ea94a742048c4b07b8f52c4310ef2" candidate_uuid: "0d0914db9cd1404ca86a8404d1916c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6dc38b0fb02468db92da7cae2f63ce1"
I20260928 16:01:44.830509 7982 raft_consensus.cc:3037] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:44.830559 8249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3ea94a742048c4b07b8f52c4310ef2" candidate_uuid: "0d0914db9cd1404ca86a8404d1916c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ccd7886e4454d9b996423e6e269beda"
I20260928 16:01:44.830777 8249 raft_consensus.cc:3037] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:44.832839 7982 raft_consensus.cc:2454] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d0914db9cd1404ca86a8404d1916c70 in term 1.
I20260928 16:01:44.833094 8249 raft_consensus.cc:2454] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d0914db9cd1404ca86a8404d1916c70 in term 1.
I20260928 16:01:44.833256 8052 leader_election.cc:302] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d0914db9cd1404ca86a8404d1916c70, d6dc38b0fb02468db92da7cae2f63ce1; no voters:
I20260928 16:01:44.833564 8318 raft_consensus.cc:2781] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:44.833993 8318 raft_consensus.cc:686] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 1 LEADER]: Becoming Leader. State: Replica: 0d0914db9cd1404ca86a8404d1916c70, State: Running, Role: LEADER
I20260928 16:01:44.834438 8318 consensus_queue.cc:237] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } }
I20260928 16:01:44.839140 7837 catalog_manager.cc:5495] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d0914db9cd1404ca86a8404d1916c70 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "0d0914db9cd1404ca86a8404d1916c70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0914db9cd1404ca86a8404d1916c70" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 } health_report { overall_health: UNKNOWN } } }
I20260928 16:01:44.854672 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.849196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.22.125.194:34717)
0928 16:01:44.849409 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.849414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.849432 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:01:44.849632 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.849636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.849711 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.849870 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:01:44.850252 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.850981 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.852540 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.852962 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.853102 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.853585 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:01:44.854078 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:01:44.854255 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.854305 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.854399 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:01:44.862761 7982 raft_consensus.cc:1261] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 1 FOLLOWER]: Refusing update from remote peer 0d0914db9cd1404ca86a8404d1916c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:44.862759 8249 raft_consensus.cc:1261] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 1 FOLLOWER]: Refusing update from remote peer 0d0914db9cd1404ca86a8404d1916c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:44.863929 8323 consensus_queue.cc:1035] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6dc38b0fb02468db92da7cae2f63ce1" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:44.864362 8318 consensus_queue.cc:1035] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ccd7886e4454d9b996423e6e269beda" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:44.877135 8327 mvcc.cc:205] Tried to move back new op lower bound from 7334343904707530752 to 7334343904605495296. Current Snapshot: MvccSnapshot[applied={T|T < 7334343904707530752}]
I20260928 16:01:44.878254 8328 mvcc.cc:205] Tried to move back new op lower bound from 7334343904707530752 to 7334343904605495296. Current Snapshot: MvccSnapshot[applied={T|T < 7334343904707530752}]
I20260928 16:01:44.890799 7836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:44.891081 7836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:44.894814 7836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70ceeaadc7154e43aade849152ed473b: Sending DeleteTablet for 3 replicas of tablet bc3ea94a742048c4b07b8f52c4310ef2
I20260928 16:01:44.895974 8096 tablet_service.cc:1514] Processing DeleteTablet for tablet bc3ea94a742048c4b07b8f52c4310ef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:44 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:57126
I20260928 16:01:44.896062 7962 tablet_service.cc:1514] Processing DeleteTablet for tablet bc3ea94a742048c4b07b8f52c4310ef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:44 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:59910
I20260928 16:01:44.896461 8229 tablet_service.cc:1514] Processing DeleteTablet for tablet bc3ea94a742048c4b07b8f52c4310ef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:44 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:38364
I20260928 16:01:44.898702 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 7898
I20260928 16:01:44.924561 8337 tablet_replica.cc:330] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: stopping tablet replica
I20260928 16:01:44.927455 8337 raft_consensus.cc:2229] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:44.928212 8337 raft_consensus.cc:2258] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:44.931859 8337 ts_tablet_manager.cc:1905] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:44.940567 8336 tablet_replica.cc:330] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: stopping tablet replica
I20260928 16:01:44.941018 8336 raft_consensus.cc:2229] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:01:44.941758 8336 raft_consensus.cc:2258] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:44.943747 8336 ts_tablet_manager.cc:1905] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:44.947183 8335 tablet_replica.cc:330] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: stopping tablet replica
Sep 28 16:01:44 dist-test-slave-9cqd krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 16:01:44.947675 8335 raft_consensus.cc:2229] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:44.948334 8335 raft_consensus.cc:2258] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:44.948505 8337 ts_tablet_manager.cc:1918] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:44.948684 8337 log.cc:1196] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/bc3ea94a742048c4b07b8f52c4310ef2
I20260928 16:01:44.949245 8337 ts_tablet_manager.cc:1939] T bc3ea94a742048c4b07b8f52c4310ef2 P 5ccd7886e4454d9b996423e6e269beda: Deleting consensus metadata
I20260928 16:01:44.950977 8335 ts_tablet_manager.cc:1905] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:44.951151 7821 catalog_manager.cc:4841] TS 5ccd7886e4454d9b996423e6e269beda (127.22.125.195:44947): tablet bc3ea94a742048c4b07b8f52c4310ef2 (table test-table [id=7f39919cffff461d9cee2963a64d3e14]) successfully deleted
I20260928 16:01:44.955639 8336 ts_tablet_manager.cc:1918] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:44.955832 8336 log.cc:1196] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/bc3ea94a742048c4b07b8f52c4310ef2
I20260928 16:01:44.956382 8336 ts_tablet_manager.cc:1939] T bc3ea94a742048c4b07b8f52c4310ef2 P 0d0914db9cd1404ca86a8404d1916c70: Deleting consensus metadata
I20260928 16:01:44.956517 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:44.900226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59926 (local address 127.22.125.193:39425)
0928 16:01:44.900403 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:44.900409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:44.924172 (+ 23763us) server_negotiation.cc:407] Connection header received
0928 16:01:44.925038 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:44.925045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:44.925143 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:44.925224 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:44.928633 (+ 3409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.929860 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.940481 (+ 10621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:44.941493 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:44.941626 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:44.948589 (+ 6963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:44.948612 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:44.948633 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:44.948675 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:44.952710 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.953662 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.953673 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.953676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.953832 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:44.954284 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:44.954289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:44.954293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:44.954762 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:44.955073 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:44.955701 (+ 628us) server_negotiation.cc:299] Negotiation successful
0928 16:01:44.956036 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:01:44.958202 8022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:44.958648 7824 catalog_manager.cc:4841] TS 0d0914db9cd1404ca86a8404d1916c70 (127.22.125.194:34717): tablet bc3ea94a742048c4b07b8f52c4310ef2 (table test-table [id=7f39919cffff461d9cee2963a64d3e14]) successfully deleted
I20260928 16:01:45.048066 8335 ts_tablet_manager.cc:1918] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:45.048362 8335 log.cc:1196] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/bc3ea94a742048c4b07b8f52c4310ef2
I20260928 16:01:45.049018 8335 ts_tablet_manager.cc:1939] T bc3ea94a742048c4b07b8f52c4310ef2 P d6dc38b0fb02468db92da7cae2f63ce1: Deleting consensus metadata
I20260928 16:01:45.049021 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 7898
I20260928 16:01:45.067268 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8033
W20260928 16:01:45.067195 7824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.125.193:39425 (error 108)
Sep 28 16:01:45 dist-test-slave-9cqd krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 16:01:45.088385 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.068340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.22.125.194:34717)
0928 16:01:45.068515 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:45.068522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:45.068770 (+ 248us) server_negotiation.cc:407] Connection header received
0928 16:01:45.069010 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:45.069016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:45.069118 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:45.069193 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:45.070940 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:45.072122 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:45.074033 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:45.074720 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:45.074810 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:45.080264 (+ 5454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:45.080280 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:45.080300 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:45.080349 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:45.084771 (+ 4422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:45.085715 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:45.085725 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:45.085729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:45.085926 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:45.086408 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:45.086413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:45.086417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:45.086874 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:45.087157 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:45.087752 (+ 595us) server_negotiation.cc:299] Negotiation successful
0928 16:01:45.087976 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:01:45.089910 8156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:45.091220 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.090646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59934)
0928 16:01:45.090860 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.091010 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260928 16:01:45.091830 7824 catalog_manager.cc:4584] TS d6dc38b0fb02468db92da7cae2f63ce1 (127.22.125.193:39425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc3ea94a742048c4b07b8f52c4310ef2: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
I20260928 16:01:45.145987 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.145379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59936)
0928 16:01:45.145670 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.145788 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260928 16:01:45.147090 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.146717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59952)
0928 16:01:45.146872 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.146956 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 16:01:45.207639 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.207082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59968)
0928 16:01:45.207317 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.207445 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 16:01:45.208822 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.208408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59978)
0928 16:01:45.208584 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.208683 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260928 16:01:45.212172 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8033
I20260928 16:01:45.231551 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8166
Sep 28 16:01:45 dist-test-slave-9cqd krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 16:01:45.245783 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.232283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38378 (local address 127.22.125.195:44947)
0928 16:01:45.232454 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:45.232459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:45.232499 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:01:45.232662 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:45.232666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:45.232727 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:45.232818 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:45.234066 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:45.234766 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:45.236011 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:45.236471 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:45.236563 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:45.240807 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:45.240821 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:45.240837 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:45.240868 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:45.243392 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:45.244015 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:45.244023 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:45.244025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:45.244214 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:45.244569 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:45.244572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:45.244574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:45.244911 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:45.245113 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:45.245391 (+ 278us) server_negotiation.cc:299] Negotiation successful
0928 16:01:45.245550 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:01:45.246577 8289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:45.289525 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.289045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59982)
0928 16:01:45.289267 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.289372 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260928 16:01:45.290628 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.290235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:59990)
0928 16:01:45.290467 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.290534 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260928 16:01:45.335983 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8166
I20260928 16:01:45.352563 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 7805
I20260928 16:01:45.353737 7867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:45.464982 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 7805
I20260928 16:01:45.465042 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.464515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39425 (local address 127.0.0.1:60000)
0928 16:01:45.464759 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:45.464870 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:39425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
2026-09-28T16:01:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4178 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T16:01:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:01:45Z Disabled control of system clock
I20260928 16:01:45.541400 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34979
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:34979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:01:45.861724 8347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:45.861999 8347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:45.862073 8347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:45.868918 8347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:01:45.869009 8347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:45.869045 8347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:45.869099 8347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:01:45.869140 8347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:01:45.877863 8347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:34979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:45.878989 8347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:45.880788 8347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:45.892122 8355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:45.892177 8353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:45.892207 8352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:45.893224 8347 server_base.cc:1034] running on GCE node
I20260928 16:01:45.893985 8347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:45.895210 8347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:45.896409 8347 hybrid_clock.cc:648] HybridClock initialized: now 1790611305896370 us; error 25 us; skew 500 ppm
I20260928 16:01:45.896795 8347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:45.898911 8347 webserver.cc:460] Webserver started at http://127.22.125.254:33655/ using document root <none> and password file <none>
I20260928 16:01:45.899562 8347 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:45.899678 8347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:45.900017 8347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:45.902868 8347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "cfd29107ce4b40baa2fd461bc540ec4b"
format_stamp: "Formatted at 2026-09-28 16:01:45 on dist-test-slave-9cqd"
server_key: "8b9b4a177481f743366aaf20fc48b063"
server_key_iv: "b03473e9a0b52cb9832b198c2778d29c"
server_key_version: "encryptionkey@0"
I20260928 16:01:45.903720 8347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "cfd29107ce4b40baa2fd461bc540ec4b"
format_stamp: "Formatted at 2026-09-28 16:01:45 on dist-test-slave-9cqd"
server_key: "8b9b4a177481f743366aaf20fc48b063"
server_key_iv: "b03473e9a0b52cb9832b198c2778d29c"
server_key_version: "encryptionkey@0"
I20260928 16:01:45.908550 8347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260928 16:01:45.911801 8361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:45.913065 8347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:01:45.913249 8347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "cfd29107ce4b40baa2fd461bc540ec4b"
format_stamp: "Formatted at 2026-09-28 16:01:45 on dist-test-slave-9cqd"
server_key: "8b9b4a177481f743366aaf20fc48b063"
server_key_iv: "b03473e9a0b52cb9832b198c2778d29c"
server_key_version: "encryptionkey@0"
I20260928 16:01:45.913457 8347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:45.932655 8347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:45.933399 8347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:45.933676 8347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:45.951990 8347 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:34979
I20260928 16:01:45.952014 8412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:34979 every 8 connection(s)
I20260928 16:01:45.953316 8347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:01:45.956907 8413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:45.961242 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8347
I20260928 16:01:45.961428 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:01:45.961891 23031 external_mini_cluster.cc:1428] Setting key a1b1603d5eabdd691c40850ad6629a49
I20260928 16:01:45.969105 8413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b: Bootstrap starting.
I20260928 16:01:45.972661 8413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:45.973960 8413 log.cc:824] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:45.976665 8413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b: No bootstrap required, opened a new log
I20260928 16:01:45.983428 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:45.963736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.22.125.254:34979)
0928 16:01:45.964884 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:45.964899 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:45.964950 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:01:45.967463 (+ 2513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:45.967491 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:45.968081 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:45.968521 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:45.969037 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:45.971058 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:45.972604 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:45.973656 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:45.973852 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:45.980538 (+ 6686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:45.980553 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:45.980571 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:45.981330 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:45.981434 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:45.981460 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:45.981592 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:45.981811 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 16:01:45.982156 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":392,"threads_started":1}
I20260928 16:01:45.985697 8413 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34979 } }
I20260928 16:01:45.986191 8413 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:45.986361 8413 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd29107ce4b40baa2fd461bc540ec4b, State: Initialized, Role: FOLLOWER
I20260928 16:01:45.986920 8413 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34979 } }
I20260928 16:01:45.987154 8413 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:01:45.987303 8413 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:01:45.987476 8413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:45.989898 8413 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34979 } }
I20260928 16:01:45.990374 8413 leader_election.cc:302] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfd29107ce4b40baa2fd461bc540ec4b; no voters:
I20260928 16:01:45.990906 8413 leader_election.cc:288] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:01:45.991063 8418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:45.991762 8418 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [term 1 LEADER]: Becoming Leader. State: Replica: cfd29107ce4b40baa2fd461bc540ec4b, State: Running, Role: LEADER
I20260928 16:01:45.992295 8418 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34979 } }
I20260928 16:01:45.992986 8413 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:01:45.994791 8419 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34979 } } }
I20260928 16:01:45.995132 8419 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:45.996562 8420 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd29107ce4b40baa2fd461bc540ec4b. Latest consensus state: current_term: 1 leader_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd29107ce4b40baa2fd461bc540ec4b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34979 } } }
I20260928 16:01:45.996961 8420 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:45.997186 8423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:01:46.002108 8423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:01:46.011037 8423 catalog_manager.cc:1347] Generated new cluster ID: a4d0c21d201b4f138b3251e2987bd5fd
I20260928 16:01:46.011157 8423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:01:46.030779 8423 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:01:46.032097 8423 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:01:46.047120 8423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b: Generated new TSK 0
I20260928 16:01:46.048133 8423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:01:46.056847 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34979
--builtin_ntp_servers=127.22.125.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:01:46.402674 8437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:46.402956 8437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:46.403047 8437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:46.410161 8437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:46.410259 8437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:46.410365 8437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:01:46.419452 8437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:46.420703 8437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:46.422478 8437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:46.435570 8445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:46.435658 8442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:46.435647 8443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:46.441330 8437 server_base.cc:1034] running on GCE node
I20260928 16:01:46.441934 8437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:46.442865 8437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:46.444116 8437 hybrid_clock.cc:648] HybridClock initialized: now 1790611306444063 us; error 30 us; skew 500 ppm
I20260928 16:01:46.444432 8437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:46.446725 8437 webserver.cc:460] Webserver started at http://127.22.125.193:38509/ using document root <none> and password file <none>
I20260928 16:01:46.447415 8437 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:46.447535 8437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:46.447871 8437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:46.450876 8437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "d10fc9aa15f84183b6697ce02e7f7a5e"
format_stamp: "Formatted at 2026-09-28 16:01:46 on dist-test-slave-9cqd"
server_key: "4716a26f7da54b675598ec74e59d64c1"
server_key_iv: "404c7040498ebacfbcf87fbedc0d1edb"
server_key_version: "encryptionkey@0"
I20260928 16:01:46.451658 8437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "d10fc9aa15f84183b6697ce02e7f7a5e"
format_stamp: "Formatted at 2026-09-28 16:01:46 on dist-test-slave-9cqd"
server_key: "4716a26f7da54b675598ec74e59d64c1"
server_key_iv: "404c7040498ebacfbcf87fbedc0d1edb"
server_key_version: "encryptionkey@0"
I20260928 16:01:46.456837 8437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260928 16:01:46.460263 8451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:46.461532 8437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:46.461730 8437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "d10fc9aa15f84183b6697ce02e7f7a5e"
format_stamp: "Formatted at 2026-09-28 16:01:46 on dist-test-slave-9cqd"
server_key: "4716a26f7da54b675598ec74e59d64c1"
server_key_iv: "404c7040498ebacfbcf87fbedc0d1edb"
server_key_version: "encryptionkey@0"
I20260928 16:01:46.461969 8437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:46.476491 8437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:46.477252 8437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:46.477559 8437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:46.478474 8437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:46.479848 8437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:46.479979 8437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:46.480118 8437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:46.480211 8437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:46.513748 8437 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:36409
I20260928 16:01:46.513847 8563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:36409 every 8 connection(s)
I20260928 16:01:46.515093 8437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:01:46.523923 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8437
I20260928 16:01:46.524166 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:01:46.524794 23031 external_mini_cluster.cc:1428] Setting key 6d3c8845578f614d7fb2c65ecfb74eeb
I20260928 16:01:46.527818 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34979
--builtin_ntp_servers=127.22.125.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:01:46.530903 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:46.517808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:39085 (local address 127.22.125.254:34979)
0928 16:01:46.518274 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:46.518280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:46.519521 (+ 1241us) server_negotiation.cc:407] Connection header received
0928 16:01:46.521650 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:46.521658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:46.521759 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:46.521908 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:46.523620 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.525005 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.527389 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.528063 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.528201 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:46.530049 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:46.530055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:46.530060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:46.530088 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:46.530131 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:46.530143 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:46.530241 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:46.530473 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 16:01:46.530573 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":237,"threads_started":1}
I20260928 16:01:46.531981 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:46.518484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34979 (local address 127.22.125.193:39085)
0928 16:01:46.519356 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:46.519392 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:46.521274 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:46.522219 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:46.522230 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:46.522366 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:46.523400 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:46.523426 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.525229 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.525234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:46.527208 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:46.527225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.528530 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.528540 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:46.528868 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:46.529702 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:01:46.529706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:46.529765 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:01:46.529851 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:46.530240 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:46.530260 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:46.530271 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 16:01:46.530463 (+ 192us) client_negotiation.cc:240] Negotiation successful
0928 16:01:46.530881 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":264,"threads_started":1}
I20260928 16:01:46.533644 8564 heartbeater.cc:347] Connected to a master server at 127.22.125.254:34979
I20260928 16:01:46.533952 8564 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:46.534690 8564 heartbeater.cc:511] Master 127.22.125.254:34979 requested a full tablet report, sending...
I20260928 16:01:46.536711 8378 ts_manager.cc:194] Registered new tserver with Master: d10fc9aa15f84183b6697ce02e7f7a5e (127.22.125.193:36409)
I20260928 16:01:46.538187 8378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.193:39085
W20260928 16:01:46.850510 8568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:46.850836 8568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:46.851009 8568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:46.858239 8568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:46.858379 8568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:46.858556 8568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:01:46.867758 8568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:46.869035 8568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:46.870819 8568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:46.883492 8573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:46.883553 8574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:46.883731 8576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:46.884377 8568 server_base.cc:1034] running on GCE node
I20260928 16:01:46.885003 8568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:46.885892 8568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:46.887120 8568 hybrid_clock.cc:648] HybridClock initialized: now 1790611306887068 us; error 62 us; skew 500 ppm
I20260928 16:01:46.887426 8568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:46.889674 8568 webserver.cc:460] Webserver started at http://127.22.125.194:45539/ using document root <none> and password file <none>
I20260928 16:01:46.890331 8568 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:46.890460 8568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:46.890762 8568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:46.893603 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "89762e118db249839361a34a620aad89"
format_stamp: "Formatted at 2026-09-28 16:01:46 on dist-test-slave-9cqd"
server_key: "6a7159616d56708d8bf4be73431f5292"
server_key_iv: "cb819835cc28880c5ca4be377bb02088"
server_key_version: "encryptionkey@0"
I20260928 16:01:46.894380 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "89762e118db249839361a34a620aad89"
format_stamp: "Formatted at 2026-09-28 16:01:46 on dist-test-slave-9cqd"
server_key: "6a7159616d56708d8bf4be73431f5292"
server_key_iv: "cb819835cc28880c5ca4be377bb02088"
server_key_version: "encryptionkey@0"
I20260928 16:01:46.899400 8568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260928 16:01:46.903028 8582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:46.904424 8568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:01:46.904587 8568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "89762e118db249839361a34a620aad89"
format_stamp: "Formatted at 2026-09-28 16:01:46 on dist-test-slave-9cqd"
server_key: "6a7159616d56708d8bf4be73431f5292"
server_key_iv: "cb819835cc28880c5ca4be377bb02088"
server_key_version: "encryptionkey@0"
I20260928 16:01:46.904800 8568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:46.923215 8568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:46.923996 8568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:46.924355 8568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:46.925269 8568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:46.926671 8568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:46.926757 8568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:46.926885 8568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:46.926988 8568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:46.959295 8568 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:43399
I20260928 16:01:46.959347 8694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:43399 every 8 connection(s)
I20260928 16:01:46.961097 8568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:01:46.962769 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8568
I20260928 16:01:46.962960 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:01:46.963500 23031 external_mini_cluster.cc:1428] Setting key 405b734b477c5aa7a1de9459693578b8
I20260928 16:01:46.966724 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34979
--builtin_ntp_servers=127.22.125.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:01:46.976763 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:46.963801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:36347 (local address 127.22.125.254:34979)
0928 16:01:46.964013 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:46.964018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:46.965794 (+ 1776us) server_negotiation.cc:407] Connection header received
0928 16:01:46.968379 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:46.968385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:46.968463 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:46.968533 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:46.970117 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.971312 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.973679 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.974390 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.974533 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:46.975820 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:46.975829 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:46.975836 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:46.975860 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:46.975901 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:46.975918 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:46.976009 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:46.976274 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 16:01:46.976401 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260928 16:01:46.977550 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:46.964492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34979 (local address 127.22.125.194:36347)
0928 16:01:46.965634 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:46.965687 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:46.968076 (+ 2389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:46.969049 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:46.969059 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:46.969164 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:46.969922 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:46.969946 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.971476 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.971482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:46.973470 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:46.973484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:46.974596 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:46.974601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:46.974805 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:46.975503 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:01:46.975506 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:46.975542 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:01:46.975615 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:46.976012 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:46.976020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:46.976025 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 16:01:46.976222 (+ 197us) client_negotiation.cc:240] Negotiation successful
0928 16:01:46.976510 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":317,"threads_started":1}
I20260928 16:01:46.978931 8695 heartbeater.cc:347] Connected to a master server at 127.22.125.254:34979
I20260928 16:01:46.979243 8695 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:46.980141 8695 heartbeater.cc:511] Master 127.22.125.254:34979 requested a full tablet report, sending...
I20260928 16:01:46.981564 8378 ts_manager.cc:194] Registered new tserver with Master: 89762e118db249839361a34a620aad89 (127.22.125.194:43399)
I20260928 16:01:46.982347 8378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.194:36347
W20260928 16:01:47.289486 8698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:47.289778 8698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:47.289964 8698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:47.297118 8698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:47.297227 8698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:47.297403 8698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:01:47.306255 8698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:34979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:47.307416 8698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:47.309157 8698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:47.321619 8703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:47.321666 8704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:47.321838 8706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:47.322822 8698 server_base.cc:1034] running on GCE node
I20260928 16:01:47.323355 8698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:47.324240 8698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:47.325435 8698 hybrid_clock.cc:648] HybridClock initialized: now 1790611307325402 us; error 28 us; skew 500 ppm
I20260928 16:01:47.325755 8698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:47.327868 8698 webserver.cc:460] Webserver started at http://127.22.125.195:39737/ using document root <none> and password file <none>
I20260928 16:01:47.328589 8698 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:47.328704 8698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:47.329025 8698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:47.331758 8698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "871319832d1e4db29437f3adc5dab5ad"
format_stamp: "Formatted at 2026-09-28 16:01:47 on dist-test-slave-9cqd"
server_key: "a1dcb2efaf7364cca000400e9a58b723"
server_key_iv: "bc43e795588699c2d9d2574c62c3d2c4"
server_key_version: "encryptionkey@0"
I20260928 16:01:47.332576 8698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "871319832d1e4db29437f3adc5dab5ad"
format_stamp: "Formatted at 2026-09-28 16:01:47 on dist-test-slave-9cqd"
server_key: "a1dcb2efaf7364cca000400e9a58b723"
server_key_iv: "bc43e795588699c2d9d2574c62c3d2c4"
server_key_version: "encryptionkey@0"
I20260928 16:01:47.337280 8698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260928 16:01:47.340503 8712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:47.341720 8698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 16:01:47.341908 8698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "871319832d1e4db29437f3adc5dab5ad"
format_stamp: "Formatted at 2026-09-28 16:01:47 on dist-test-slave-9cqd"
server_key: "a1dcb2efaf7364cca000400e9a58b723"
server_key_iv: "bc43e795588699c2d9d2574c62c3d2c4"
server_key_version: "encryptionkey@0"
I20260928 16:01:47.342115 8698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:47.362179 8698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:47.362926 8698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:47.363220 8698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:47.364168 8698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:47.365824 8698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:47.365994 8698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:47.366132 8698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:47.366216 8698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:47.400666 8698 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:42855
I20260928 16:01:47.400753 8824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:42855 every 8 connection(s)
I20260928 16:01:47.401928 8698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:01:47.402343 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8698
I20260928 16:01:47.402529 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:01:47.403151 23031 external_mini_cluster.cc:1428] Setting key 8bf698c585594ee68a2a6a24b0729d09
I20260928 16:01:47.414490 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.404744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:35093 (local address 127.22.125.254:34979)
0928 16:01:47.404927 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.404931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.406798 (+ 1867us) server_negotiation.cc:407] Connection header received
0928 16:01:47.408691 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.408695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.408776 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.408897 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:47.409875 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.410621 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.412122 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.412514 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.412605 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.413872 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:47.413876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:47.413879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:47.413897 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:47.413922 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:47.413933 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:47.413999 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.414187 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.414257 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:01:47.415437 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.405426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:34979 (local address 127.22.125.195:35093)
0928 16:01:47.406642 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:47.406686 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:47.408463 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:47.409113 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:47.409120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:47.409202 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:47.409739 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.409755 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.410753 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.410757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.411957 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.411969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.412687 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.412692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.412899 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.413591 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:01:47.413593 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:47.413630 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:01:47.413703 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:47.414016 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:47.414023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:47.414029 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 16:01:47.414161 (+ 132us) client_negotiation.cc:240] Negotiation successful
0928 16:01:47.414430 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":515,"threads_started":1}
I20260928 16:01:47.416841 8825 heartbeater.cc:347] Connected to a master server at 127.22.125.254:34979
I20260928 16:01:47.417150 8825 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:47.417941 8825 heartbeater.cc:511] Master 127.22.125.254:34979 requested a full tablet report, sending...
I20260928 16:01:47.419257 8378 ts_manager.cc:194] Registered new tserver with Master: 871319832d1e4db29437f3adc5dab5ad (127.22.125.195:42855)
I20260928 16:01:47.419989 8378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.195:35093
I20260928 16:01:47.430253 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:01:47.439728 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.434108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.22.125.254:34979)
0928 16:01:47.434334 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.434343 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.434679 (+ 336us) server_negotiation.cc:407] Connection header received
0928 16:01:47.434805 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.434809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.434877 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.434954 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:47.435375 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.436111 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.437385 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.437783 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.437869 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.439038 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:47.439044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:47.439049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:47.439075 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:47.439124 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:47.439142 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:47.439222 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.439368 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.439453 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:01:47.443292 8378 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:01:47.450495 8378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:01:47.484215 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.461964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50444 (local address 127.22.125.195:42855)
0928 16:01:47.476620 (+ 14656us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.476628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.476661 (+ 33us) server_negotiation.cc:407] Connection header received
0928 16:01:47.476766 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.476775 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.477017 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.477263 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:47.477931 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.479326 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.482000 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.482967 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.483188 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.483434 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.483614 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.483785 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14491,"thread_start_us":14340,"threads_started":1}
I20260928 16:01:47.484236 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.461412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:42855 (local address 127.0.0.1:50444)
0928 16:01:47.462621 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:47.462663 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:47.462934 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:47.477248 (+ 14314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:47.477253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:47.477278 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:47.477712 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.477725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.479482 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.479489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.481740 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.481753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.483165 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.483172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.483362 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.483412 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 16:01:47.483574 (+ 162us) client_negotiation.cc:240] Negotiation successful
0928 16:01:47.483785 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937,"mutex_wait_us":254,"thread_start_us":12993,"threads_started":1}
I20260928 16:01:47.484596 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.461559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:43399 (local address 127.0.0.1:37556)
0928 16:01:47.474037 (+ 12478us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:47.474061 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:47.474202 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:47.476615 (+ 2413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:47.476620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:47.476648 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:47.476994 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.477006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.479526 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.479532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.481933 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.481946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.483680 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.483686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.483876 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.483923 (+ 47us) client_negotiation.cc:769] Sending connection context
0928 16:01:47.483981 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 16:01:47.484138 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12302,"mutex_wait_us":26,"thread_start_us":748,"threads_started":1}
I20260928 16:01:47.484891 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.463354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46522 (local address 127.22.125.193:36409)
0928 16:01:47.473382 (+ 10028us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.473394 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.475160 (+ 1766us) server_negotiation.cc:407] Connection header received
0928 16:01:47.475301 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.475313 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.475621 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.475873 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:47.476844 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.478707 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.481919 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.483381 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.483523 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.483758 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.484348 (+ 590us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.484521 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9814,"thread_start_us":10025,"threads_started":1}
I20260928 16:01:47.485570 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.463080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:36409 (local address 127.0.0.1:46522)
0928 16:01:47.474863 (+ 11783us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:47.474886 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:47.475020 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:47.476169 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:47.476176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:47.476207 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:47.476599 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.476614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.478856 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.478861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.481651 (+ 2790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.481669 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.483936 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.483942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.484148 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.484181 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 16:01:47.485144 (+ 963us) client_negotiation.cc:240] Negotiation successful
0928 16:01:47.485266 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11506,"mutex_wait_us":204,"thread_start_us":10641,"threads_started":1}
I20260928 16:01:47.485751 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.462441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.22.125.194:43399)
0928 16:01:47.475986 (+ 13545us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.475992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.476015 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:01:47.476146 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.476154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.476394 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.476603 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:47.477181 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.479378 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.482136 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.483471 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.484968 (+ 1497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.485175 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.485236 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.485405 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13388,"thread_start_us":13248,"threads_started":1}
I20260928 16:01:47.489038 8630 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 (DEFAULT_TABLE table=test-table [id=5f99bf2db93c48ae8e9cf705dd25c6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:47.489038 8760 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 (DEFAULT_TABLE table=test-table [id=5f99bf2db93c48ae8e9cf705dd25c6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:47.489400 8499 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 (DEFAULT_TABLE table=test-table [id=5f99bf2db93c48ae8e9cf705dd25c6f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:47.490530 8760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:47.490530 8630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:47.490726 8499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:47.501883 8843 tablet_bootstrap.cc:492] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Bootstrap starting.
I20260928 16:01:47.505265 8844 tablet_bootstrap.cc:492] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Bootstrap starting.
I20260928 16:01:47.505753 8843 tablet_bootstrap.cc:654] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:47.507120 8843 log.cc:824] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:47.508900 8844 tablet_bootstrap.cc:654] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:47.510067 8844 log.cc:824] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:47.510404 8843 tablet_bootstrap.cc:492] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: No bootstrap required, opened a new log
I20260928 16:01:47.510690 8843 ts_tablet_manager.cc:1397] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260928 16:01:47.512871 8844 tablet_bootstrap.cc:492] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: No bootstrap required, opened a new log
I20260928 16:01:47.513106 8844 ts_tablet_manager.cc:1397] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 16:01:47.515550 8845 tablet_bootstrap.cc:492] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Bootstrap starting.
I20260928 16:01:47.519002 8845 tablet_bootstrap.cc:654] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:47.520202 8845 log.cc:824] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:47.522535 8843 raft_consensus.cc:348] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.523061 8843 raft_consensus.cc:374] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:47.523224 8843 raft_consensus.cc:729] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d10fc9aa15f84183b6697ce02e7f7a5e, State: Initialized, Role: FOLLOWER
I20260928 16:01:47.523942 8843 consensus_queue.cc:260] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.524781 8844 raft_consensus.cc:348] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.525390 8844 raft_consensus.cc:374] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:47.525578 8844 raft_consensus.cc:729] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871319832d1e4db29437f3adc5dab5ad, State: Initialized, Role: FOLLOWER
I20260928 16:01:47.526436 8844 consensus_queue.cc:260] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.528064 8564 heartbeater.cc:503] Master 127.22.125.254:34979 was elected leader, sending a full tablet report...
I20260928 16:01:47.528614 8843 ts_tablet_manager.cc:1428] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Time spent starting tablet: real 0.018s user 0.012s sys 0.006s
I20260928 16:01:47.529407 8825 heartbeater.cc:503] Master 127.22.125.254:34979 was elected leader, sending a full tablet report...
I20260928 16:01:47.530084 8844 ts_tablet_manager.cc:1428] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
I20260928 16:01:47.532294 8845 tablet_bootstrap.cc:492] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: No bootstrap required, opened a new log
I20260928 16:01:47.532577 8845 ts_tablet_manager.cc:1397] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.006s
I20260928 16:01:47.540238 8845 raft_consensus.cc:348] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.540580 8845 raft_consensus.cc:374] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:47.540689 8845 raft_consensus.cc:729] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89762e118db249839361a34a620aad89, State: Initialized, Role: FOLLOWER
I20260928 16:01:47.541173 8845 consensus_queue.cc:260] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.542080 8695 heartbeater.cc:503] Master 127.22.125.254:34979 was elected leader, sending a full tablet report...
I20260928 16:01:47.542449 8845 ts_tablet_manager.cc:1428] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
W20260928 16:01:47.653672 8826 tablet.cc:2369] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:01:47.713235 8696 tablet.cc:2369] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:01:47.723902 8851 raft_consensus.cc:482] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:47.724218 8851 raft_consensus.cc:504] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.725934 8851 leader_election.cc:288] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 871319832d1e4db29437f3adc5dab5ad (127.22.125.195:42855), d10fc9aa15f84183b6697ce02e7f7a5e (127.22.125.193:36409)
I20260928 16:01:47.733064 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.726372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:49319 (local address 127.22.125.195:42855)
0928 16:01:47.726586 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.726595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.727605 (+ 1010us) server_negotiation.cc:407] Connection header received
0928 16:01:47.727811 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.727816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.727872 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.727963 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:47.728681 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.729374 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.731258 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.732135 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.732307 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.732375 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.732686 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.732813 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 16:01:47.733144 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.726555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:48411 (local address 127.22.125.193:36409)
0928 16:01:47.726797 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.726803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.727532 (+ 729us) server_negotiation.cc:407] Connection header received
0928 16:01:47.727765 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.727770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.727832 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.727933 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:47.728577 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.729270 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.731344 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.732214 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.732414 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.732480 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.732755 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.732884 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 16:01:47.733213 8853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.726377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:42855 (local address 127.22.125.194:49319)
0928 16:01:47.727491 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:47.727506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:47.727660 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:47.727990 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:47.727995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:47.728017 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:47.728522 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.728530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.729518 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.729523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.731073 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.731085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.732319 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.732324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.732534 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.732557 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 16:01:47.732661 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 16:01:47.732834 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"mutex_wait_us":47,"thread_start_us":783,"threads_started":1}
I20260928 16:01:47.733260 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.726275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:36409 (local address 127.22.125.194:48411)
0928 16:01:47.727419 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:47.727443 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:47.727604 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:47.727954 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:47.727959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:47.727983 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:47.728419 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.728428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.729416 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.729423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.731101 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:47.731122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.732440 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.732447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:47.732597 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.732618 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 16:01:47.732759 (+ 141us) client_negotiation.cc:240] Negotiation successful
0928 16:01:47.732890 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":111,"thread_start_us":570,"threads_started":1}
I20260928 16:01:47.734423 8519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c5bc7f09e4fa9bab4c4fc26b78a25" candidate_uuid: "89762e118db249839361a34a620aad89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" is_pre_election: true
I20260928 16:01:47.734421 8780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c5bc7f09e4fa9bab4c4fc26b78a25" candidate_uuid: "89762e118db249839361a34a620aad89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871319832d1e4db29437f3adc5dab5ad" is_pre_election: true
I20260928 16:01:47.734897 8519 raft_consensus.cc:2454] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89762e118db249839361a34a620aad89 in term 0.
I20260928 16:01:47.734910 8780 raft_consensus.cc:2454] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89762e118db249839361a34a620aad89 in term 0.
I20260928 16:01:47.735526 8585 leader_election.cc:302] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89762e118db249839361a34a620aad89, d10fc9aa15f84183b6697ce02e7f7a5e; no voters:
I20260928 16:01:47.735932 8851 raft_consensus.cc:2781] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:01:47.736104 8851 raft_consensus.cc:482] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:01:47.736213 8851 raft_consensus.cc:3037] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:47.738250 8851 raft_consensus.cc:504] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.738778 8851 leader_election.cc:288] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [CANDIDATE]: Term 1 election: Requested vote from peers 871319832d1e4db29437f3adc5dab5ad (127.22.125.195:42855), d10fc9aa15f84183b6697ce02e7f7a5e (127.22.125.193:36409)
I20260928 16:01:47.739209 8780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c5bc7f09e4fa9bab4c4fc26b78a25" candidate_uuid: "89762e118db249839361a34a620aad89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871319832d1e4db29437f3adc5dab5ad"
I20260928 16:01:47.739219 8519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c5bc7f09e4fa9bab4c4fc26b78a25" candidate_uuid: "89762e118db249839361a34a620aad89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e"
I20260928 16:01:47.739419 8780 raft_consensus.cc:3037] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:47.739418 8519 raft_consensus.cc:3037] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:47.742645 8519 raft_consensus.cc:2454] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89762e118db249839361a34a620aad89 in term 1.
I20260928 16:01:47.742645 8780 raft_consensus.cc:2454] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89762e118db249839361a34a620aad89 in term 1.
I20260928 16:01:47.743062 8584 leader_election.cc:302] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 871319832d1e4db29437f3adc5dab5ad, 89762e118db249839361a34a620aad89; no voters:
I20260928 16:01:47.743358 8851 raft_consensus.cc:2781] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:47.743777 8851 raft_consensus.cc:686] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 1 LEADER]: Becoming Leader. State: Replica: 89762e118db249839361a34a620aad89, State: Running, Role: LEADER
I20260928 16:01:47.744283 8851 consensus_queue.cc:237] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } }
I20260928 16:01:47.749070 8378 catalog_manager.cc:5495] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89762e118db249839361a34a620aad89 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "89762e118db249839361a34a620aad89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89762e118db249839361a34a620aad89" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 } health_report { overall_health: UNKNOWN } } }
W20260928 16:01:47.767603 8565 tablet.cc:2369] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:01:47.776079 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.770540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.22.125.194:43399)
0928 16:01:47.770751 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.770758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.770861 (+ 103us) server_negotiation.cc:407] Connection header received
0928 16:01:47.771094 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.771099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.771188 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.771334 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:01:47.771702 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.772479 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.774002 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.774414 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.774555 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.775021 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:01:47.775495 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:01:47.775653 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.775710 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.775784 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:01:47.784183 8519 raft_consensus.cc:1261] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 1 FOLLOWER]: Refusing update from remote peer 89762e118db249839361a34a620aad89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:47.784184 8780 raft_consensus.cc:1261] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 1 FOLLOWER]: Refusing update from remote peer 89762e118db249839361a34a620aad89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:47.785107 8854 consensus_queue.cc:1035] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "871319832d1e4db29437f3adc5dab5ad" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:47.785470 8851 consensus_queue.cc:1035] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d10fc9aa15f84183b6697ce02e7f7a5e" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 36409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:47.800686 8859 mvcc.cc:205] Tried to move back new op lower bound from 7334343916673413120 to 7334343916524265472. Current Snapshot: MvccSnapshot[applied={T|T < 7334343916673413120}]
I20260928 16:01:47.810508 8378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:47.810775 8378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:47.814254 8378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfd29107ce4b40baa2fd461bc540ec4b: Sending DeleteTablet for 3 replicas of tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25
I20260928 16:01:47.815399 8760 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:47 UTC) from {username='slave'} at 127.0.0.1:50444
I20260928 16:01:47.815498 8499 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:47 UTC) from {username='slave'} at 127.0.0.1:46522
I20260928 16:01:47.816208 8630 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:47 UTC) from {username='slave'} at 127.0.0.1:37556
I20260928 16:01:47.816884 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8437
I20260928 16:01:47.826997 8868 tablet_replica.cc:330] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: stopping tablet replica
I20260928 16:01:47.827186 8867 tablet_replica.cc:330] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: stopping tablet replica
I20260928 16:01:47.827454 8868 raft_consensus.cc:2229] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:47.827538 8867 raft_consensus.cc:2229] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:47.828236 8867 raft_consensus.cc:2258] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:47.828570 8868 raft_consensus.cc:2258] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:47.830578 8867 ts_tablet_manager.cc:1905] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:47.830968 8868 ts_tablet_manager.cc:1905] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:47.838193 8868 ts_tablet_manager.cc:1918] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:47.838402 8868 log.cc:1196] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/9a7c5bc7f09e4fa9bab4c4fc26b78a25
I20260928 16:01:47.839038 8868 ts_tablet_manager.cc:1939] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P d10fc9aa15f84183b6697ce02e7f7a5e: Deleting consensus metadata
I20260928 16:01:47.840412 8869 tablet_replica.cc:330] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: stopping tablet replica
I20260928 16:01:47.840845 8869 raft_consensus.cc:2229] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:01:47.840998 8364 catalog_manager.cc:4841] TS d10fc9aa15f84183b6697ce02e7f7a5e (127.22.125.193:36409): tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 (table test-table [id=5f99bf2db93c48ae8e9cf705dd25c6f7]) successfully deleted
I20260928 16:01:47.841449 8869 raft_consensus.cc:2258] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:47.843542 8869 ts_tablet_manager.cc:1905] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:47.843983 8867 ts_tablet_manager.cc:1918] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:47.844278 8867 log.cc:1196] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/9a7c5bc7f09e4fa9bab4c4fc26b78a25
I20260928 16:01:47.844817 8867 ts_tablet_manager.cc:1939] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 871319832d1e4db29437f3adc5dab5ad: Deleting consensus metadata
I20260928 16:01:47.846066 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:47.818553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.22.125.193:36409)
0928 16:01:47.818788 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:47.818797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:47.830753 (+ 11956us) server_negotiation.cc:407] Connection header received
0928 16:01:47.830927 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:47.830936 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:47.831064 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:47.831186 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:47.838515 (+ 7329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.840082 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.841923 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:47.842559 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:47.842692 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:47.844246 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:47.844259 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:47.844275 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:47.845013 (+ 738us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:47.845074 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:47.845105 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:47.845192 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:47.845573 (+ 381us) server_negotiation.cc:299] Negotiation successful
0928 16:01:47.845688 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260928 16:01:47.846776 8363 catalog_manager.cc:4841] TS 871319832d1e4db29437f3adc5dab5ad (127.22.125.195:42855): tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 (table test-table [id=5f99bf2db93c48ae8e9cf705dd25c6f7]) successfully deleted
I20260928 16:01:47.847245 8559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:47.852375 8869 ts_tablet_manager.cc:1918] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:47.852563 8869 log.cc:1196] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/9a7c5bc7f09e4fa9bab4c4fc26b78a25
I20260928 16:01:47.853032 8869 ts_tablet_manager.cc:1939] T 9a7c5bc7f09e4fa9bab4c4fc26b78a25 P 89762e118db249839361a34a620aad89: Deleting consensus metadata
I20260928 16:01:47.854435 8364 catalog_manager.cc:4841] TS 89762e118db249839361a34a620aad89 (127.22.125.194:43399): tablet 9a7c5bc7f09e4fa9bab4c4fc26b78a25 (table test-table [id=5f99bf2db93c48ae8e9cf705dd25c6f7]) successfully deleted
I20260928 16:01:47.998689 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8437
I20260928 16:01:48.023480 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8568
I20260928 16:01:48.030900 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:48.024449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.22.125.194:43399)
0928 16:01:48.024649 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:48.024655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:48.024676 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:01:48.024776 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:48.024781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:48.024897 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:48.025011 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:48.025445 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:48.026278 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:48.027576 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:48.028027 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:48.028207 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:48.029385 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:48.029406 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:48.029420 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:48.030043 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:48.030090 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:48.030110 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:48.030227 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:48.030388 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 16:01:48.030488 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 16:01:48.031610 8690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:48.141434 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8568
I20260928 16:01:48.159538 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8698
I20260928 16:01:48.165869 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:48.160344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.22.125.195:42855)
0928 16:01:48.160511 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:48.160515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:48.160534 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:01:48.160698 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:48.160702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:48.160793 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:48.160884 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:48.161247 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:48.161954 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:48.163178 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:48.163544 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:48.163633 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:48.164658 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:48.164666 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:48.164680 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:48.165250 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:48.165295 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:48.165320 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:48.165417 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:48.165516 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 16:01:48.165602 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 16:01:48.166566 8820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:48.254935 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8698
I20260928 16:01:48.271731 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8347
I20260928 16:01:48.272854 8408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:48.383623 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8347
2026-09-28T16:01:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2915 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:01:48 dist-test-slave-9cqd krb5kdc[8877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:01:48 dist-test-slave-9cqd krb5kdc[8877](info): set up 2 sockets
Sep 28 16:01:48 dist-test-slave-9cqd krb5kdc[8877](info): commencing operation
krb5kdc: starting...
W20260928 16:01:49.489979 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:01:49 dist-test-slave-9cqd krb5kdc[8877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:01:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:01:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:01:49.718993 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:41417
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:41417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:50.037290 8890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:50.037595 8890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:50.037683 8890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:50.044768 8890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:01:50.044866 8890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:50.044972 8890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:50.045070 8890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:01:50.045148 8890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:01:50.054601 8890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:41417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:41417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:50.055910 8890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:50.057408 8890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:50.069105 8896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:50.069239 8895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:50.069216 8898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:50.070048 8890 server_base.cc:1034] running on GCE node
I20260928 16:01:50.070840 8890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:50.072326 8890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:50.073555 8890 hybrid_clock.cc:648] HybridClock initialized: now 1790611310073518 us; error 49 us; skew 500 ppm
I20260928 16:01:50.073895 8890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:50 dist-test-slave-9cqd krb5kdc[8877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611310, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:50.076902 8890 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:01:50.078379 8890 webserver.cc:460] Webserver started at http://127.22.125.254:41947/ using document root <none> and password file <none>
I20260928 16:01:50.079041 8890 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:50.079187 8890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:50.079497 8890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:50.082515 8890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "3d1e9e7b8aa443929605e885d4c07b97"
format_stamp: "Formatted at 2026-09-28 16:01:50 on dist-test-slave-9cqd"
server_key: "fec5b7a169891b1c481feeb26d8ab7aa"
server_key_iv: "1bddcc4f2890f343285a830c1fb242a9"
server_key_version: "encryptionkey@0"
I20260928 16:01:50.083278 8890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "3d1e9e7b8aa443929605e885d4c07b97"
format_stamp: "Formatted at 2026-09-28 16:01:50 on dist-test-slave-9cqd"
server_key: "fec5b7a169891b1c481feeb26d8ab7aa"
server_key_iv: "1bddcc4f2890f343285a830c1fb242a9"
server_key_version: "encryptionkey@0"
I20260928 16:01:50.088403 8890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260928 16:01:50.091742 8905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:50.093052 8890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:01:50.093238 8890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "3d1e9e7b8aa443929605e885d4c07b97"
format_stamp: "Formatted at 2026-09-28 16:01:50 on dist-test-slave-9cqd"
server_key: "fec5b7a169891b1c481feeb26d8ab7aa"
server_key_iv: "1bddcc4f2890f343285a830c1fb242a9"
server_key_version: "encryptionkey@0"
I20260928 16:01:50.093456 8890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:50.109613 8890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:50.113452 8890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:50.113739 8890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:50.131934 8890 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:41417
I20260928 16:01:50.131955 8956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:41417 every 8 connection(s)
I20260928 16:01:50.133317 8890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:01:50.136891 8957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:50.139815 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8890
I20260928 16:01:50.140026 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:01:50.140611 23031 external_mini_cluster.cc:1428] Setting key d4ef9d8b43a331366235c49847a09d80
I20260928 16:01:50.149165 8957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97: Bootstrap starting.
I20260928 16:01:50.152951 8957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:50.154294 8957 log.cc:824] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:01:50 dist-test-slave-9cqd krb5kdc[8877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:50.158239 8957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97: No bootstrap required, opened a new log
I20260928 16:01:50.165331 8960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:50.142497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.22.125.254:41417)
0928 16:01:50.143245 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:50.143264 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:50.143305 (+ 41us) server_negotiation.cc:407] Connection header received
0928 16:01:50.145018 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:50.145040 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:50.145441 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:50.145757 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:50.147567 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:50.148927 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:50.150747 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:50.151416 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:50.151539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:50.156701 (+ 5162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:50.156711 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:50.156730 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:50.156776 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:50.160783 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:50.161502 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:50.161511 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:50.161519 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:50.161732 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:50.162096 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:50.162105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:50.162110 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:50.162809 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:50.163117 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:50.163464 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 16:01:50.163917 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":353,"threads_started":1}
I20260928 16:01:50.168141 8957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d1e9e7b8aa443929605e885d4c07b97" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41417 } }
I20260928 16:01:50.168462 8957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:50.168533 8957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d1e9e7b8aa443929605e885d4c07b97, State: Initialized, Role: FOLLOWER
I20260928 16:01:50.169065 8957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d1e9e7b8aa443929605e885d4c07b97" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41417 } }
I20260928 16:01:50.169270 8957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:01:50.169370 8957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:01:50.169539 8957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:50.171924 8957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d1e9e7b8aa443929605e885d4c07b97" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41417 } }
I20260928 16:01:50.172415 8957 leader_election.cc:302] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d1e9e7b8aa443929605e885d4c07b97; no voters:
I20260928 16:01:50.172948 8957 leader_election.cc:288] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:01:50.173048 8962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:50.173780 8962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [term 1 LEADER]: Becoming Leader. State: Replica: 3d1e9e7b8aa443929605e885d4c07b97, State: Running, Role: LEADER
I20260928 16:01:50.174424 8962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d1e9e7b8aa443929605e885d4c07b97" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41417 } }
I20260928 16:01:50.175120 8957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:01:50.177397 8964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d1e9e7b8aa443929605e885d4c07b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d1e9e7b8aa443929605e885d4c07b97" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41417 } } }
I20260928 16:01:50.177901 8964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:50.178524 8963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d1e9e7b8aa443929605e885d4c07b97. Latest consensus state: current_term: 1 leader_uuid: "3d1e9e7b8aa443929605e885d4c07b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d1e9e7b8aa443929605e885d4c07b97" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 41417 } } }
I20260928 16:01:50.185535 8963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97 [sys.catalog]: This master's current role is: LEADER
W20260928 16:01:50.193300 8978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:01:50.193444 8978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:01:50.193612 8969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:01:50.195218 8969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:01:50.202993 8969 catalog_manager.cc:1347] Generated new cluster ID: 87a4843011e44c50ad3e133a94f69568
I20260928 16:01:50.203190 8969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:01:50.212540 8969 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:01:50.214306 8969 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:01:50.236886 8969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97: Generated new TSK 0
I20260928 16:01:50.237922 8969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 16:01:50.301445 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41417
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:50.631870 8983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:50.632189 8983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:50.632375 8983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:50.639150 8983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:50.639240 8983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:50.639384 8983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:01:50.648875 8983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:50.650080 8983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:50.651327 8983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:50.664325 8989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:50.664634 8988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:50.665210 8991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:50.665233 8983 server_base.cc:1034] running on GCE node
I20260928 16:01:50.665990 8983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:50.667124 8983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:50.668313 8983 hybrid_clock.cc:648] HybridClock initialized: now 1790611310668255 us; error 52 us; skew 500 ppm
I20260928 16:01:50.668617 8983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:50 dist-test-slave-9cqd krb5kdc[8877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611310, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:50.671563 8983 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:01:50.672968 8983 webserver.cc:460] Webserver started at http://127.22.125.193:41871/ using document root <none> and password file <none>
I20260928 16:01:50.673602 8983 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:50.673738 8983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:50.674054 8983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:50.676918 8983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "7ad12ad26a224330aefcbd68a6e5113b"
format_stamp: "Formatted at 2026-09-28 16:01:50 on dist-test-slave-9cqd"
server_key: "3f2e733f49da5c7ef6af722dd3ce2d7f"
server_key_iv: "89e51b56cad1ecf299a6e06611c54d4d"
server_key_version: "encryptionkey@0"
I20260928 16:01:50.677631 8983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "7ad12ad26a224330aefcbd68a6e5113b"
format_stamp: "Formatted at 2026-09-28 16:01:50 on dist-test-slave-9cqd"
server_key: "3f2e733f49da5c7ef6af722dd3ce2d7f"
server_key_iv: "89e51b56cad1ecf299a6e06611c54d4d"
server_key_version: "encryptionkey@0"
I20260928 16:01:50.682686 8983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:01:50.686174 8998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:50.687434 8983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:01:50.687594 8983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "7ad12ad26a224330aefcbd68a6e5113b"
format_stamp: "Formatted at 2026-09-28 16:01:50 on dist-test-slave-9cqd"
server_key: "3f2e733f49da5c7ef6af722dd3ce2d7f"
server_key_iv: "89e51b56cad1ecf299a6e06611c54d4d"
server_key_version: "encryptionkey@0"
I20260928 16:01:50.687786 8983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:50.707957 8983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:50.711813 8983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:50.712124 8983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:50.713102 8983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:50.714423 8983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:50.714499 8983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:50.714673 8983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:50.714735 8983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:50.747640 8983 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:35733
I20260928 16:01:50.747725 9110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:35733 every 8 connection(s)
I20260928 16:01:50.748963 8983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:01:50.753898 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 8983
I20260928 16:01:50.754091 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:01:50.754527 23031 external_mini_cluster.cc:1428] Setting key 1504591563f07654dc855807f9e40755
Sep 28 16:01:50 dist-test-slave-9cqd krb5kdc[8877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611310, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:50.768832 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:50.751379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:59153 (local address 127.22.125.254:41417)
0928 16:01:50.751829 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:50.751835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:50.752815 (+ 980us) server_negotiation.cc:407] Connection header received
0928 16:01:50.754704 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:50.754709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:50.754805 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:50.754938 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:50.756681 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:50.757413 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:50.759032 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:50.759448 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:50.759582 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:50.763196 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:50.763200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:50.763204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:50.763235 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:50.765831 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:50.766577 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:50.766582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:50.766584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:50.766698 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:50.767126 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:50.767131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:50.767136 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:50.767408 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:50.767561 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:50.768258 (+ 697us) server_negotiation.cc:299] Negotiation successful
0928 16:01:50.768486 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":213,"threads_started":1}
I20260928 16:01:50.769750 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:50.751921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:41417 (local address 127.22.125.193:59153)
0928 16:01:50.752656 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:50.752692 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:50.754435 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:50.755175 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:50.755208 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:50.755818 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:50.756487 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:50.756503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:50.757571 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:50.757579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:50.758874 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:50.758886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:50.759685 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:50.759692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:50.759920 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:50.760663 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:50.760689 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:50.762972 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:50.765982 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:50.765991 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:50.766004 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:50.766441 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:50.766816 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:50.766821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:50.766824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:50.767006 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:50.767533 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:50.767540 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:50.767847 (+ 307us) client_negotiation.cc:769] Sending connection context
0928 16:01:50.768147 (+ 300us) client_negotiation.cc:240] Negotiation successful
0928 16:01:50.768618 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":317,"threads_started":1}
I20260928 16:01:50.771409 9111 heartbeater.cc:347] Connected to a master server at 127.22.125.254:41417
I20260928 16:01:50.771697 9111 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:50.772410 9111 heartbeater.cc:511] Master 127.22.125.254:41417 requested a full tablet report, sending...
I20260928 16:01:50.774675 8922 ts_manager.cc:194] Registered new tserver with Master: 7ad12ad26a224330aefcbd68a6e5113b (127.22.125.193:35733)
I20260928 16:01:50.776894 8922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:59153
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 16:01:50.807504 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41417
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:51.127593 9117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:51.127863 9117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:51.127957 9117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:51.135191 9117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:51.135288 9117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:51.135462 9117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:01:51.145484 9117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:51.146680 9117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:51.147946 9117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:51.161057 9122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:51.161083 9125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:51.161118 9123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:51.161883 9117 server_base.cc:1034] running on GCE node
I20260928 16:01:51.162514 9117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:51.163391 9117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:51.164615 9117 hybrid_clock.cc:648] HybridClock initialized: now 1790611311164567 us; error 58 us; skew 500 ppm
I20260928 16:01:51.164923 9117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:51 dist-test-slave-9cqd krb5kdc[8877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611311, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:51.172401 9117 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:01:51.173681 9117 webserver.cc:460] Webserver started at http://127.22.125.194:41193/ using document root <none> and password file <none>
I20260928 16:01:51.174300 9117 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:51.174398 9117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:51.174642 9117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:51.177496 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "b85dfc747b974159b6fc1142239da838"
format_stamp: "Formatted at 2026-09-28 16:01:51 on dist-test-slave-9cqd"
server_key: "05a70f8f482b5396c422bc8046d0ad55"
server_key_iv: "14ad3ad850f55777f3dd29cbb8a463cb"
server_key_version: "encryptionkey@0"
I20260928 16:01:51.178253 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "b85dfc747b974159b6fc1142239da838"
format_stamp: "Formatted at 2026-09-28 16:01:51 on dist-test-slave-9cqd"
server_key: "05a70f8f482b5396c422bc8046d0ad55"
server_key_iv: "14ad3ad850f55777f3dd29cbb8a463cb"
server_key_version: "encryptionkey@0"
I20260928 16:01:51.183248 9117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260928 16:01:51.186616 9132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:51.187804 9117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:01:51.187980 9117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "b85dfc747b974159b6fc1142239da838"
format_stamp: "Formatted at 2026-09-28 16:01:51 on dist-test-slave-9cqd"
server_key: "05a70f8f482b5396c422bc8046d0ad55"
server_key_iv: "14ad3ad850f55777f3dd29cbb8a463cb"
server_key_version: "encryptionkey@0"
I20260928 16:01:51.188235 9117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:51.210678 9117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:51.214407 9117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:51.214694 9117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:51.215626 9117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:51.216920 9117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:51.216991 9117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:51.217136 9117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:51.217196 9117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:51.249876 9117 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:35073
I20260928 16:01:51.249963 9244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:35073 every 8 connection(s)
I20260928 16:01:51.251170 9117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:01:51.260664 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 9117
I20260928 16:01:51.260857 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:01:51.261319 23031 external_mini_cluster.cc:1428] Setting key 2f8d25a5620179bcee0896aa6cfa877f
Sep 28 16:01:51 dist-test-slave-9cqd krb5kdc[8877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611311, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:51.271754 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.253663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:59157 (local address 127.22.125.254:41417)
0928 16:01:51.253870 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:51.253877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:51.255070 (+ 1193us) server_negotiation.cc:407] Connection header received
0928 16:01:51.256803 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:51.256807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:51.256876 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:51.256967 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:51.258403 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.259390 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.260932 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.261386 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.261525 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.265844 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:51.265853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:51.265860 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:51.265899 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:51.268571 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:51.269505 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:51.269508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:51.269510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:51.269627 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:51.270087 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:51.270090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:51.270092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:51.270354 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:51.270463 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:51.271345 (+ 882us) server_negotiation.cc:299] Negotiation successful
0928 16:01:51.271516 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:01:51.272897 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.254213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:41417 (local address 127.22.125.194:59157)
0928 16:01:51.254907 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:51.254945 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:51.256560 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:51.257175 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:51.257183 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:51.257682 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:51.258254 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.258271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.259519 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.259523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.260755 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.260772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.261619 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.261627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.261925 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.262630 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:51.262658 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:51.265604 (+ 2946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:51.268709 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:51.268722 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:51.268741 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:51.269387 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:51.269734 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:51.269741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:51.269747 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:51.269977 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:51.270491 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:51.270500 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:51.270941 (+ 441us) client_negotiation.cc:769] Sending connection context
0928 16:01:51.271236 (+ 295us) client_negotiation.cc:240] Negotiation successful
0928 16:01:51.271801 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":255,"threads_started":1}
I20260928 16:01:51.274384 9245 heartbeater.cc:347] Connected to a master server at 127.22.125.254:41417
I20260928 16:01:51.274727 9245 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:51.275528 9245 heartbeater.cc:511] Master 127.22.125.254:41417 requested a full tablet report, sending...
I20260928 16:01:51.277118 8922 ts_manager.cc:194] Registered new tserver with Master: b85dfc747b974159b6fc1142239da838 (127.22.125.194:35073)
I20260928 16:01:51.278363 8922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:59157
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 16:01:51.317204 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41417
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:51.637781 9250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:51.638087 9250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:51.638262 9250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:51.645169 9250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:51.645309 9250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:51.645486 9250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:01:51.654645 9250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:51.655857 9250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:51.657150 9250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:51.670189 9258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:51.670266 9255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:51.670235 9256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:51.671000 9250 server_base.cc:1034] running on GCE node
I20260928 16:01:51.671533 9250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:51.672549 9250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:51.673763 9250 hybrid_clock.cc:648] HybridClock initialized: now 1790611311673718 us; error 49 us; skew 500 ppm
I20260928 16:01:51.674058 9250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:51 dist-test-slave-9cqd krb5kdc[8877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611311, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:51.677189 9250 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:01:51.678632 9250 webserver.cc:460] Webserver started at http://127.22.125.195:37383/ using document root <none> and password file <none>
I20260928 16:01:51.679273 9250 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:51.679418 9250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:51.679751 9250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:51.682725 9250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "8fbe9ae6265345fe856436b4482688a5"
format_stamp: "Formatted at 2026-09-28 16:01:51 on dist-test-slave-9cqd"
server_key: "3242e307ddf96a07bfc27b4d9a821e84"
server_key_iv: "137f886df9b50aa219aa5c9a85430188"
server_key_version: "encryptionkey@0"
I20260928 16:01:51.683454 9250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "8fbe9ae6265345fe856436b4482688a5"
format_stamp: "Formatted at 2026-09-28 16:01:51 on dist-test-slave-9cqd"
server_key: "3242e307ddf96a07bfc27b4d9a821e84"
server_key_iv: "137f886df9b50aa219aa5c9a85430188"
server_key_version: "encryptionkey@0"
I20260928 16:01:51.688414 9250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260928 16:01:51.691592 9265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:51.693092 9250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:01:51.693257 9250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "8fbe9ae6265345fe856436b4482688a5"
format_stamp: "Formatted at 2026-09-28 16:01:51 on dist-test-slave-9cqd"
server_key: "3242e307ddf96a07bfc27b4d9a821e84"
server_key_iv: "137f886df9b50aa219aa5c9a85430188"
server_key_version: "encryptionkey@0"
I20260928 16:01:51.693461 9250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:51.715126 9250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:51.718819 9250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:51.719099 9250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:51.720008 9250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:51.721310 9250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:51.721413 9250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:51.721558 9250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:51.721642 9250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:51.754490 9250 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:39437
I20260928 16:01:51.754572 9377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:39437 every 8 connection(s)
I20260928 16:01:51.755784 9250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:01:51.759460 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 9250
I20260928 16:01:51.759646 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:01:51.760370 23031 external_mini_cluster.cc:1428] Setting key 1868c92df7d3402d95e85167b0a834ae
Sep 28 16:01:51 dist-test-slave-9cqd krb5kdc[8877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611311, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:51.775205 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.758230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:45521 (local address 127.22.125.254:41417)
0928 16:01:51.758408 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:51.758413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:51.759515 (+ 1102us) server_negotiation.cc:407] Connection header received
0928 16:01:51.761741 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:51.761746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:51.761857 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:51.761959 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:51.763603 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.764424 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.765848 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.766245 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.766351 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.769521 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:51.769525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:51.769529 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:51.769559 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:51.772132 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:51.772732 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:51.772736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:51.772738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:51.772869 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:51.773208 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:51.773211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:51.773213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:51.773582 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:51.773715 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:51.774583 (+ 868us) server_negotiation.cc:299] Negotiation successful
0928 16:01:51.774817 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:01:51.776144 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.758665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:41417 (local address 127.22.125.195:45521)
0928 16:01:51.759364 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:51.759398 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:51.761426 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:51.762257 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:51.762265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:51.762797 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:51.763456 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.763473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.764538 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.764541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.765728 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.765739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.766431 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.766435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.766613 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.767213 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:51.767234 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:51.769350 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:51.772237 (+ 2887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:51.772243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:51.772253 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:51.772639 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:51.772957 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:51.772960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:51.772962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:51.773119 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:51.773728 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:51.773735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:51.774125 (+ 390us) client_negotiation.cc:769] Sending connection context
0928 16:01:51.774451 (+ 326us) client_negotiation.cc:240] Negotiation successful
0928 16:01:51.774947 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":252,"threads_started":1}
I20260928 16:01:51.777455 9378 heartbeater.cc:347] Connected to a master server at 127.22.125.254:41417
I20260928 16:01:51.777738 9378 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:51.778391 9378 heartbeater.cc:511] Master 127.22.125.254:41417 requested a full tablet report, sending...
I20260928 16:01:51.779716 8922 ts_manager.cc:194] Registered new tserver with Master: 8fbe9ae6265345fe856436b4482688a5 (127.22.125.195:39437)
I20260928 16:01:51.779794 9111 heartbeater.cc:503] Master 127.22.125.254:41417 was elected leader, sending a full tablet report...
I20260928 16:01:51.780807 8922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:45521
I20260928 16:01:51.787139 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:01:51.803359 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.791262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.22.125.254:41417)
0928 16:01:51.791449 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:51.791454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:51.791907 (+ 453us) server_negotiation.cc:407] Connection header received
0928 16:01:51.792016 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:51.792019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:51.792123 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:51.792223 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:51.793641 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.794316 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.795517 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.795920 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.796047 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.798741 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:51.798745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:51.798748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:51.798777 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:51.801079 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:51.801768 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:51.801772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:51.801774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:51.801897 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:51.802254 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:51.802257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:51.802259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:51.802523 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:51.802718 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:51.803012 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 16:01:51.803129 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:01:51.806901 8922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:01:51.813506 8922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:01:51.842597 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.824905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.22.125.195:39437)
0928 16:01:51.834040 (+ 9135us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:51.834048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:51.834092 (+ 44us) server_negotiation.cc:407] Connection header received
0928 16:01:51.834202 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:51.834210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:51.834466 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:51.834688 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:51.835435 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.836976 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.839981 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.841387 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.841580 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.841769 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:51.842062 (+ 293us) server_negotiation.cc:299] Negotiation successful
0928 16:01:51.842226 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8948,"thread_start_us":8803,"threads_started":1}
I20260928 16:01:51.842744 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.825518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.22.125.194:35073)
0928 16:01:51.833353 (+ 7835us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:51.833361 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:51.834219 (+ 858us) server_negotiation.cc:407] Connection header received
0928 16:01:51.834353 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:51.834359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:51.834544 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:51.834718 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:51.835941 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.837513 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.840149 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.841575 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.841752 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.841908 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:51.842109 (+ 201us) server_negotiation.cc:299] Negotiation successful
0928 16:01:51.842395 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7659,"thread_start_us":7572,"threads_started":1}
I20260928 16:01:51.842864 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.824671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:39437 (local address 127.0.0.1:47908)
0928 16:01:51.832082 (+ 7411us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:51.832129 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:51.832300 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:51.834700 (+ 2400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:51.834708 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:51.834736 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:51.835225 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.835242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.837132 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.837137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.839800 (+ 2663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.839820 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.841608 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.841615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.841834 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.841870 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 16:01:51.841998 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 16:01:51.842386 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7168,"mutex_wait_us":152,"thread_start_us":7031,"threads_started":1}
I20260928 16:01:51.843219 9394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.825520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:35733 (local address 127.0.0.1:45696)
0928 16:01:51.833883 (+ 8363us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:51.833902 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:51.834043 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:51.835143 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:51.835147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:51.835175 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:51.835705 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.835720 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.837413 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.837417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.839264 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.839276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.841335 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.841342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.841584 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.842446 (+ 862us) client_negotiation.cc:769] Sending connection context
0928 16:01:51.842536 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 16:01:51.842755 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8244,"mutex_wait_us":838,"thread_start_us":8003,"threads_started":1}
I20260928 16:01:51.843914 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.825876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.22.125.193:35733)
0928 16:01:51.833010 (+ 7134us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:51.833022 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:51.834027 (+ 1005us) server_negotiation.cc:407] Connection header received
0928 16:01:51.834674 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:51.834681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:51.834930 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:51.835134 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:51.835958 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.837247 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.839425 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.840811 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.841000 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.841183 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:51.843512 (+ 2329us) server_negotiation.cc:299] Negotiation successful
0928 16:01:51.843683 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6961,"thread_start_us":6800,"threads_started":1}
I20260928 16:01:51.844388 9391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:51.824960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:35073 (local address 127.0.0.1:46694)
0928 16:01:51.834094 (+ 9134us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:51.834120 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:51.834230 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:51.834991 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:51.834998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:51.835019 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:51.835711 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.835720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.837671 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.837677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.839965 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:51.839976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:51.841789 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:51.841797 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:51.841919 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:51.841941 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 16:01:51.842056 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 16:01:51.842273 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8476,"mutex_wait_us":6,"thread_start_us":8332,"threads_started":1}
I20260928 16:01:51.847149 9313 tablet_service.cc:1467] Processing CreateTablet for tablet 63495a37f5d9452bb82b2dd247e648da (DEFAULT_TABLE table=test-table [id=551fe49f15164c9c96d243bfe75d0d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:51.847668 9046 tablet_service.cc:1467] Processing CreateTablet for tablet 63495a37f5d9452bb82b2dd247e648da (DEFAULT_TABLE table=test-table [id=551fe49f15164c9c96d243bfe75d0d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:51.848541 9313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63495a37f5d9452bb82b2dd247e648da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:51.848321 9180 tablet_service.cc:1467] Processing CreateTablet for tablet 63495a37f5d9452bb82b2dd247e648da (DEFAULT_TABLE table=test-table [id=551fe49f15164c9c96d243bfe75d0d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:51.849107 9046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63495a37f5d9452bb82b2dd247e648da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:51.849617 9180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63495a37f5d9452bb82b2dd247e648da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:51.859511 9396 tablet_bootstrap.cc:492] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Bootstrap starting.
I20260928 16:01:51.862634 9396 tablet_bootstrap.cc:654] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:51.863947 9396 log.cc:824] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:51.864943 9398 tablet_bootstrap.cc:492] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Bootstrap starting.
I20260928 16:01:51.863407 9397 tablet_bootstrap.cc:492] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Bootstrap starting.
I20260928 16:01:51.867638 9396 tablet_bootstrap.cc:492] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: No bootstrap required, opened a new log
I20260928 16:01:51.867908 9396 ts_tablet_manager.cc:1397] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260928 16:01:51.868412 9398 tablet_bootstrap.cc:654] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:51.868409 9397 tablet_bootstrap.cc:654] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:51.869580 9397 log.cc:824] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:51.869577 9398 log.cc:824] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:51.872151 9398 tablet_bootstrap.cc:492] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: No bootstrap required, opened a new log
I20260928 16:01:51.872387 9398 ts_tablet_manager.cc:1397] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 16:01:51.872619 9397 tablet_bootstrap.cc:492] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: No bootstrap required, opened a new log
I20260928 16:01:51.872869 9397 ts_tablet_manager.cc:1397] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260928 16:01:51.877899 9396 raft_consensus.cc:348] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:51.878322 9396 raft_consensus.cc:374] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:51.878424 9396 raft_consensus.cc:729] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fbe9ae6265345fe856436b4482688a5, State: Initialized, Role: FOLLOWER
I20260928 16:01:51.879042 9396 consensus_queue.cc:260] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:51.880201 9378 heartbeater.cc:503] Master 127.22.125.254:41417 was elected leader, sending a full tablet report...
I20260928 16:01:51.880376 9398 raft_consensus.cc:348] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:51.880759 9398 raft_consensus.cc:374] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:51.880846 9398 raft_consensus.cc:729] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85dfc747b974159b6fc1142239da838, State: Initialized, Role: FOLLOWER
I20260928 16:01:51.880849 9396 ts_tablet_manager.cc:1428] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260928 16:01:51.881527 9398 consensus_queue.cc:260] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:51.882520 9245 heartbeater.cc:503] Master 127.22.125.254:41417 was elected leader, sending a full tablet report...
I20260928 16:01:51.882907 9397 raft_consensus.cc:348] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:51.883293 9397 raft_consensus.cc:374] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:51.883423 9397 raft_consensus.cc:729] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad12ad26a224330aefcbd68a6e5113b, State: Initialized, Role: FOLLOWER
I20260928 16:01:51.884023 9397 consensus_queue.cc:260] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:51.884410 9398 ts_tablet_manager.cc:1428] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260928 16:01:51.886211 9397 ts_tablet_manager.cc:1428] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260928 16:01:52.001263 9112 tablet.cc:2369] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:01:52.003084 9246 tablet.cc:2369] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 16:01:52.007285 9379 tablet.cc:2369] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:01:52.265820 9404 raft_consensus.cc:482] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:52.266069 9404 raft_consensus.cc:504] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:52.267627 9404 leader_election.cc:288] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fbe9ae6265345fe856436b4482688a5 (127.22.125.195:39437), b85dfc747b974159b6fc1142239da838 (127.22.125.194:35073)
I20260928 16:01:52.277056 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.268199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:60425 (local address 127.22.125.194:35073)
0928 16:01:52.268427 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:52.268435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:52.269196 (+ 761us) server_negotiation.cc:407] Connection header received
0928 16:01:52.269435 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:52.269441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:52.269514 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:52.269613 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:52.270567 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.271696 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.274667 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.275949 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.276115 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.276188 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:52.276716 (+ 528us) server_negotiation.cc:299] Negotiation successful
0928 16:01:52.276881 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 16:01:52.277133 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.268259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:34173 (local address 127.22.125.195:39437)
0928 16:01:52.268438 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:52.268444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:52.269247 (+ 803us) server_negotiation.cc:407] Connection header received
0928 16:01:52.269435 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:52.269441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:52.269514 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:52.269613 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:52.270567 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.271696 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.274671 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.275932 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.276116 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.276188 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:52.276764 (+ 576us) server_negotiation.cc:299] Negotiation successful
0928 16:01:52.276892 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:01:52.277298 9406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.267955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:35073 (local address 127.22.125.193:60425)
0928 16:01:52.269072 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:52.269093 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:52.269284 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:52.269646 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:52.269651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:52.269715 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:52.270405 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:52.270413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.271830 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.271833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:52.274505 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:52.274521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.276230 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.276242 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:52.276499 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.276568 (+ 69us) client_negotiation.cc:769] Sending connection context
0928 16:01:52.276673 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 16:01:52.276816 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":294,"spinlock_wait_cycles":41600,"thread_start_us":694,"threads_started":1}
I20260928 16:01:52.277712 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.268125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:39437 (local address 127.22.125.193:34173)
0928 16:01:52.269072 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:52.269093 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:52.269248 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:52.269646 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:52.269651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:52.269713 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:52.270392 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:52.270406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.271828 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.271833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:52.274505 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:52.274521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.276258 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.276264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:52.276492 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.276606 (+ 114us) client_negotiation.cc:769] Sending connection context
0928 16:01:52.277321 (+ 715us) client_negotiation.cc:240] Negotiation successful
0928 16:01:52.277447 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":332,"spinlock_wait_cycles":14464,"thread_start_us":524,"threads_started":1}
I20260928 16:01:52.278252 9200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63495a37f5d9452bb82b2dd247e648da" candidate_uuid: "7ad12ad26a224330aefcbd68a6e5113b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85dfc747b974159b6fc1142239da838" is_pre_election: true
I20260928 16:01:52.278570 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63495a37f5d9452bb82b2dd247e648da" candidate_uuid: "7ad12ad26a224330aefcbd68a6e5113b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fbe9ae6265345fe856436b4482688a5" is_pre_election: true
I20260928 16:01:52.278717 9200 raft_consensus.cc:2454] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ad12ad26a224330aefcbd68a6e5113b in term 0.
I20260928 16:01:52.279031 9333 raft_consensus.cc:2454] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ad12ad26a224330aefcbd68a6e5113b in term 0.
I20260928 16:01:52.279434 8999 leader_election.cc:302] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ad12ad26a224330aefcbd68a6e5113b, b85dfc747b974159b6fc1142239da838; no voters:
I20260928 16:01:52.279843 9404 raft_consensus.cc:2781] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:01:52.280004 9404 raft_consensus.cc:482] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:01:52.280154 9404 raft_consensus.cc:3037] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:52.282477 9404 raft_consensus.cc:504] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:52.283025 9404 leader_election.cc:288] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [CANDIDATE]: Term 1 election: Requested vote from peers 8fbe9ae6265345fe856436b4482688a5 (127.22.125.195:39437), b85dfc747b974159b6fc1142239da838 (127.22.125.194:35073)
I20260928 16:01:52.283438 9333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63495a37f5d9452bb82b2dd247e648da" candidate_uuid: "7ad12ad26a224330aefcbd68a6e5113b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fbe9ae6265345fe856436b4482688a5"
I20260928 16:01:52.283460 9200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63495a37f5d9452bb82b2dd247e648da" candidate_uuid: "7ad12ad26a224330aefcbd68a6e5113b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b85dfc747b974159b6fc1142239da838"
I20260928 16:01:52.283653 9333 raft_consensus.cc:3037] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:52.283653 9200 raft_consensus.cc:3037] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:52.286810 9333 raft_consensus.cc:2454] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ad12ad26a224330aefcbd68a6e5113b in term 1.
I20260928 16:01:52.286826 9200 raft_consensus.cc:2454] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ad12ad26a224330aefcbd68a6e5113b in term 1.
I20260928 16:01:52.287276 9002 leader_election.cc:302] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ad12ad26a224330aefcbd68a6e5113b, 8fbe9ae6265345fe856436b4482688a5; no voters:
I20260928 16:01:52.287663 9404 raft_consensus.cc:2781] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:52.288225 9404 raft_consensus.cc:686] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 1 LEADER]: Becoming Leader. State: Replica: 7ad12ad26a224330aefcbd68a6e5113b, State: Running, Role: LEADER
I20260928 16:01:52.288756 9404 consensus_queue.cc:237] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } }
I20260928 16:01:52.293624 8922 catalog_manager.cc:5495] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ad12ad26a224330aefcbd68a6e5113b (127.22.125.193). New cstate: current_term: 1 leader_uuid: "7ad12ad26a224330aefcbd68a6e5113b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ad12ad26a224330aefcbd68a6e5113b" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 35733 } health_report { overall_health: HEALTHY } } }
I20260928 16:01:52.410059 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.404772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.22.125.193:35733)
0928 16:01:52.405227 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:52.405232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:52.405249 (+ 17us) server_negotiation.cc:407] Connection header received
0928 16:01:52.405445 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:52.405448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:52.405526 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:52.405673 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:01:52.406056 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.406739 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.408133 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.408530 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.408621 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.409058 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:01:52.409547 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:01:52.409688 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:52.409811 (+ 123us) server_negotiation.cc:299] Negotiation successful
0928 16:01:52.409851 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260928 16:01:52.417757 9333 raft_consensus.cc:1261] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 1 FOLLOWER]: Refusing update from remote peer 7ad12ad26a224330aefcbd68a6e5113b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:52.417757 9200 raft_consensus.cc:1261] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 1 FOLLOWER]: Refusing update from remote peer 7ad12ad26a224330aefcbd68a6e5113b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:52.418810 9407 consensus_queue.cc:1035] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85dfc747b974159b6fc1142239da838" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 35073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:52.419173 9404 consensus_queue.cc:1035] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fbe9ae6265345fe856436b4482688a5" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 39437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:52.428344 9412 mvcc.cc:205] Tried to move back new op lower bound from 7334343935652974592 to 7334343935139004416. Current Snapshot: MvccSnapshot[applied={T|T < 7334343935652974592}]
I20260928 16:01:52.445225 8921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:52.445489 8921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:52.448887 8921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d1e9e7b8aa443929605e885d4c07b97: Sending DeleteTablet for 3 replicas of tablet 63495a37f5d9452bb82b2dd247e648da
I20260928 16:01:52.449986 9313 tablet_service.cc:1514] Processing DeleteTablet for tablet 63495a37f5d9452bb82b2dd247e648da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:52 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:47908
I20260928 16:01:52.450423 9180 tablet_service.cc:1514] Processing DeleteTablet for tablet 63495a37f5d9452bb82b2dd247e648da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:52 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:46694
I20260928 16:01:52.450731 9046 tablet_service.cc:1514] Processing DeleteTablet for tablet 63495a37f5d9452bb82b2dd247e648da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:52 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:45696
I20260928 16:01:52.451057 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8983
I20260928 16:01:52.461642 9426 tablet_replica.cc:330] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: stopping tablet replica
I20260928 16:01:52.462302 9426 raft_consensus.cc:2229] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:01:52.463284 9426 raft_consensus.cc:2258] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:52.463948 9425 tablet_replica.cc:330] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: stopping tablet replica
I20260928 16:01:52.464675 9425 raft_consensus.cc:2229] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:52.465232 9425 raft_consensus.cc:2258] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:52.466349 9426 ts_tablet_manager.cc:1905] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:52.467105 9425 ts_tablet_manager.cc:1905] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:52.474035 9426 ts_tablet_manager.cc:1918] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:52.474217 9426 log.cc:1196] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/63495a37f5d9452bb82b2dd247e648da
I20260928 16:01:52.474339 9425 ts_tablet_manager.cc:1918] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:52.474495 9425 log.cc:1196] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/63495a37f5d9452bb82b2dd247e648da
I20260928 16:01:52.474848 9426 ts_tablet_manager.cc:1939] T 63495a37f5d9452bb82b2dd247e648da P 7ad12ad26a224330aefcbd68a6e5113b: Deleting consensus metadata
I20260928 16:01:52.476424 9425 ts_tablet_manager.cc:1939] T 63495a37f5d9452bb82b2dd247e648da P b85dfc747b974159b6fc1142239da838: Deleting consensus metadata
Sep 28 16:01:52 dist-test-slave-9cqd krb5kdc[8877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 16:01:52.476547 9424 tablet_replica.cc:330] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: stopping tablet replica
I20260928 16:01:52.476994 9424 raft_consensus.cc:2229] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:52.477465 9424 raft_consensus.cc:2258] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:52.478260 8906 catalog_manager.cc:4841] TS b85dfc747b974159b6fc1142239da838 (127.22.125.194:35073): tablet 63495a37f5d9452bb82b2dd247e648da (table test-table [id=551fe49f15164c9c96d243bfe75d0d98]) successfully deleted
I20260928 16:01:52.479394 8908 catalog_manager.cc:4841] TS 7ad12ad26a224330aefcbd68a6e5113b (127.22.125.193:35733): tablet 63495a37f5d9452bb82b2dd247e648da (table test-table [id=551fe49f15164c9c96d243bfe75d0d98]) successfully deleted
I20260928 16:01:52.479652 9424 ts_tablet_manager.cc:1905] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:52.485762 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.454781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.22.125.193:35733)
0928 16:01:52.455022 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:52.455031 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:52.464074 (+ 9043us) server_negotiation.cc:407] Connection header received
0928 16:01:52.464923 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:52.464929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:52.465007 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:52.465110 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:52.467445 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.468696 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.470723 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.471454 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.471579 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.478027 (+ 6448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:52.478042 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:52.478059 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:52.478105 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:52.482431 (+ 4326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:52.483310 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:52.483318 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:52.483323 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:52.483476 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:52.483913 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:52.483919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:52.483923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:52.484400 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:52.484743 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:52.485054 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 16:01:52.485318 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":64}
I20260928 16:01:52.485761 9424 ts_tablet_manager.cc:1918] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:52.485934 9424 log.cc:1196] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/63495a37f5d9452bb82b2dd247e648da
I20260928 16:01:52.486488 9424 ts_tablet_manager.cc:1939] T 63495a37f5d9452bb82b2dd247e648da P 8fbe9ae6265345fe856436b4482688a5: Deleting consensus metadata
I20260928 16:01:52.486529 9106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:52.488811 8909 catalog_manager.cc:4841] TS 8fbe9ae6265345fe856436b4482688a5 (127.22.125.195:39437): tablet 63495a37f5d9452bb82b2dd247e648da (table test-table [id=551fe49f15164c9c96d243bfe75d0d98]) successfully deleted
I20260928 16:01:52.591445 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8983
I20260928 16:01:52.609838 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 9117
Sep 28 16:01:52 dist-test-slave-9cqd krb5kdc[8877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 16:01:52.623870 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.610649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38680 (local address 127.22.125.194:35073)
0928 16:01:52.610824 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:52.610829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:52.610847 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:01:52.610970 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:52.610973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:52.611035 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:52.611115 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:52.612410 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.613143 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.614303 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.614701 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.614788 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.618886 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:52.618899 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:52.618914 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:52.618945 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:52.621669 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:52.622237 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:52.622244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:52.622246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:52.622377 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:52.622683 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:52.622686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:52.622688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:52.623030 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:52.623242 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:52.623480 (+ 238us) server_negotiation.cc:299] Negotiation successful
0928 16:01:52.623626 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:01:52.624651 9240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:52.713791 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 9117
I20260928 16:01:52.731534 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 9250
Sep 28 16:01:52 dist-test-slave-9cqd krb5kdc[8877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 16:01:52.745996 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:52.732286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50976 (local address 127.22.125.195:39437)
0928 16:01:52.732492 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:52.732496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:52.732515 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:01:52.732650 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:52.732654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:52.732716 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:52.732808 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:52.734045 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.734793 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.736002 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:52.736484 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:52.736574 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:52.740771 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:52.740786 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:52.740801 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:52.740849 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:52.743595 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:52.744302 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:52.744309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:52.744311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:52.744440 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:52.744760 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:52.744763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:52.744765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:52.745071 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:52.745285 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:52.745472 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 16:01:52.745627 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260928 16:01:52.746649 9373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:52.835810 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 9250
I20260928 16:01:52.852355 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 8890
I20260928 16:01:52.853506 8952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:52.962258 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 8890
2026-09-28T16:01:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4579 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T16:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:01:53Z Disabled control of system clock
I20260928 16:01:53.035624 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35901
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:35901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:01:53.358412 9435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:53.358686 9435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:53.358767 9435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:53.366026 9435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:01:53.366166 9435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:53.366232 9435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:53.366349 9435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:01:53.366432 9435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:01:53.375404 9435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:35901
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:53.376657 9435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:53.378425 9435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:53.390213 9440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:53.390180 9441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:53.390144 9443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:53.390938 9435 server_base.cc:1034] running on GCE node
I20260928 16:01:53.391677 9435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:53.393060 9435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:53.394256 9435 hybrid_clock.cc:648] HybridClock initialized: now 1790611313394208 us; error 38 us; skew 500 ppm
I20260928 16:01:53.394603 9435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:53.396797 9435 webserver.cc:460] Webserver started at http://127.22.125.254:42413/ using document root <none> and password file <none>
I20260928 16:01:53.397472 9435 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:53.397606 9435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:53.397923 9435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:53.400830 9435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "71f4545483e34cb4a789a43dcbc58dbb"
format_stamp: "Formatted at 2026-09-28 16:01:53 on dist-test-slave-9cqd"
server_key: "67e410f44803076994758efc6f8a6c26"
server_key_iv: "8f1566df993d8fd8d017121f32701130"
server_key_version: "encryptionkey@0"
I20260928 16:01:53.401645 9435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "71f4545483e34cb4a789a43dcbc58dbb"
format_stamp: "Formatted at 2026-09-28 16:01:53 on dist-test-slave-9cqd"
server_key: "67e410f44803076994758efc6f8a6c26"
server_key_iv: "8f1566df993d8fd8d017121f32701130"
server_key_version: "encryptionkey@0"
I20260928 16:01:53.406495 9435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:01:53.409758 9449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:53.411059 9435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:01:53.411252 9435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "71f4545483e34cb4a789a43dcbc58dbb"
format_stamp: "Formatted at 2026-09-28 16:01:53 on dist-test-slave-9cqd"
server_key: "67e410f44803076994758efc6f8a6c26"
server_key_iv: "8f1566df993d8fd8d017121f32701130"
server_key_version: "encryptionkey@0"
I20260928 16:01:53.411470 9435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:53.438519 9435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:53.439301 9435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:53.439608 9435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:53.457916 9435 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:35901
I20260928 16:01:53.457937 9500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:35901 every 8 connection(s)
I20260928 16:01:53.459231 9435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:01:53.462755 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:53.465575 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 9435
I20260928 16:01:53.465772 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:01:53.466248 23031 external_mini_cluster.cc:1428] Setting key 4dce3ade62292d43be5fa4d645a0460c
I20260928 16:01:53.475963 9501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb: Bootstrap starting.
I20260928 16:01:53.479735 9501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:53.480738 9501 log.cc:824] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:53.483290 9501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb: No bootstrap required, opened a new log
I20260928 16:01:53.489068 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:53.468143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37972 (local address 127.22.125.254:35901)
0928 16:01:53.469197 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:53.469210 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:53.469256 (+ 46us) server_negotiation.cc:407] Connection header received
0928 16:01:53.471523 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:53.471556 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:53.472162 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:53.472603 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:53.473124 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:53.475124 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:53.476833 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:53.477926 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:53.478124 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:53.486186 (+ 8062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:53.486202 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:53.486221 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:53.486963 (+ 742us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:53.487064 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:53.487090 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:53.487212 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:53.487411 (+ 199us) server_negotiation.cc:299] Negotiation successful
0928 16:01:53.487766 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":341,"threads_started":1}
I20260928 16:01:53.492664 9501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f4545483e34cb4a789a43dcbc58dbb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35901 } }
I20260928 16:01:53.492986 9501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:53.493060 9501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71f4545483e34cb4a789a43dcbc58dbb, State: Initialized, Role: FOLLOWER
I20260928 16:01:53.493571 9501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f4545483e34cb4a789a43dcbc58dbb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35901 } }
I20260928 16:01:53.493772 9501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:01:53.493875 9501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:01:53.493995 9501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:53.496390 9501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f4545483e34cb4a789a43dcbc58dbb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35901 } }
I20260928 16:01:53.496832 9501 leader_election.cc:302] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71f4545483e34cb4a789a43dcbc58dbb; no voters:
I20260928 16:01:53.497336 9501 leader_election.cc:288] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:01:53.497504 9506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:53.498255 9506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [term 1 LEADER]: Becoming Leader. State: Replica: 71f4545483e34cb4a789a43dcbc58dbb, State: Running, Role: LEADER
I20260928 16:01:53.498745 9506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f4545483e34cb4a789a43dcbc58dbb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35901 } }
I20260928 16:01:53.499346 9501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:01:53.501591 9507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71f4545483e34cb4a789a43dcbc58dbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f4545483e34cb4a789a43dcbc58dbb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35901 } } }
I20260928 16:01:53.501597 9508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71f4545483e34cb4a789a43dcbc58dbb. Latest consensus state: current_term: 1 leader_uuid: "71f4545483e34cb4a789a43dcbc58dbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f4545483e34cb4a789a43dcbc58dbb" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35901 } } }
I20260928 16:01:53.502074 9507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:53.502075 9508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:53.503024 9511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:01:53.513373 9511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:01:53.523144 9511 catalog_manager.cc:1347] Generated new cluster ID: 7f391df9bfc247ec8fe59ea0f4e205e2
I20260928 16:01:53.523304 9511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:01:53.545147 9511 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:01:53.546394 9511 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:01:53.559652 9511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb: Generated new TSK 0
I20260928 16:01:53.560559 9511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:01:53.575943 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35901
--builtin_ntp_servers=127.22.125.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:01:53.920370 9525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:53.920614 9525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:53.920760 9525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:53.927670 9525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:53.927757 9525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:53.927918 9525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:01:53.936487 9525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:53.937640 9525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:53.939270 9525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:53.952255 9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:53.952286 9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:53.952450 9533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:53.953081 9525 server_base.cc:1034] running on GCE node
I20260928 16:01:53.953584 9525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:53.954402 9525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:53.955600 9525 hybrid_clock.cc:648] HybridClock initialized: now 1790611313955563 us; error 25 us; skew 500 ppm
I20260928 16:01:53.955947 9525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:53.958034 9525 webserver.cc:460] Webserver started at http://127.22.125.193:37415/ using document root <none> and password file <none>
I20260928 16:01:53.958669 9525 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:53.958782 9525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:53.959087 9525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:53.962270 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "98b200ac578247f883fa48fc36c6770c"
format_stamp: "Formatted at 2026-09-28 16:01:53 on dist-test-slave-9cqd"
server_key: "4366fa0f207b39fd6e3e48f6439dd94b"
server_key_iv: "13688363ecb696c998f7261c1b4ac0ef"
server_key_version: "encryptionkey@0"
I20260928 16:01:53.962990 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "98b200ac578247f883fa48fc36c6770c"
format_stamp: "Formatted at 2026-09-28 16:01:53 on dist-test-slave-9cqd"
server_key: "4366fa0f207b39fd6e3e48f6439dd94b"
server_key_iv: "13688363ecb696c998f7261c1b4ac0ef"
server_key_version: "encryptionkey@0"
I20260928 16:01:53.967792 9525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260928 16:01:53.971061 9539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:53.972345 9525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:01:53.972510 9525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "98b200ac578247f883fa48fc36c6770c"
format_stamp: "Formatted at 2026-09-28 16:01:53 on dist-test-slave-9cqd"
server_key: "4366fa0f207b39fd6e3e48f6439dd94b"
server_key_iv: "13688363ecb696c998f7261c1b4ac0ef"
server_key_version: "encryptionkey@0"
I20260928 16:01:53.972707 9525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:53.985003 9525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:53.985723 9525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:53.985994 9525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:53.986898 9525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:53.988317 9525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:53.988426 9525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:53.988551 9525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:53.988662 9525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:54.021077 9525 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:32883
I20260928 16:01:54.021160 9652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:32883 every 8 connection(s)
I20260928 16:01:54.022336 9525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:01:54.031746 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 9525
I20260928 16:01:54.031975 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:01:54.032616 23031 external_mini_cluster.cc:1428] Setting key 694cd0250a5113d7441462dc69b7f361
I20260928 16:01:54.034547 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.024813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:35227 (local address 127.22.125.254:35901)
0928 16:01:54.025243 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:54.025247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:54.026427 (+ 1180us) server_negotiation.cc:407] Connection header received
0928 16:01:54.028134 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:54.028143 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:54.028254 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:54.028358 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:54.029318 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.030027 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.031489 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.032217 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.032349 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.033691 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:54.033697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:54.033700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:54.033721 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:54.033781 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:54.033797 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:54.033900 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:54.034113 (+ 213us) server_negotiation.cc:299] Negotiation successful
0928 16:01:54.034197 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":223,"threads_started":1}
I20260928 16:01:54.035663 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35901
--builtin_ntp_servers=127.22.125.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:01:54.035686 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.025415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35901 (local address 127.22.125.193:35227)
0928 16:01:54.026261 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:54.026296 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:54.027857 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:54.028554 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:54.028562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:54.028643 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:54.029171 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.029187 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.030148 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.030152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.031348 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.031359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.032419 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.032424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.032607 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.033335 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:01:54.033339 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:54.033398 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:01:54.033486 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:54.033905 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:54.033914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:54.033919 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 16:01:54.034069 (+ 150us) client_negotiation.cc:240] Negotiation successful
0928 16:01:54.034401 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":255,"threads_started":1}
I20260928 16:01:54.037129 9653 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35901
I20260928 16:01:54.037427 9653 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:54.038095 9653 heartbeater.cc:511] Master 127.22.125.254:35901 requested a full tablet report, sending...
I20260928 16:01:54.040129 9466 ts_manager.cc:194] Registered new tserver with Master: 98b200ac578247f883fa48fc36c6770c (127.22.125.193:32883)
I20260928 16:01:54.041618 9466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.193:35227
W20260928 16:01:54.357465 9657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:54.357772 9657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:54.357913 9657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:54.365217 9657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:54.365342 9657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:54.365512 9657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:01:54.374277 9657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:54.375587 9657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:54.377471 9657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:54.390367 9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:54.390489 9663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:54.390477 9662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:54.391377 9657 server_base.cc:1034] running on GCE node
I20260928 16:01:54.391961 9657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:54.392868 9657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:54.394098 9657 hybrid_clock.cc:648] HybridClock initialized: now 1790611314394043 us; error 43 us; skew 500 ppm
I20260928 16:01:54.394431 9657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:54.396682 9657 webserver.cc:460] Webserver started at http://127.22.125.194:37461/ using document root <none> and password file <none>
I20260928 16:01:54.397329 9657 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:54.397459 9657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:54.397794 9657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:54.400838 9657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "1c2d711bdf8f4fa6a073760d76d216d4"
format_stamp: "Formatted at 2026-09-28 16:01:54 on dist-test-slave-9cqd"
server_key: "3db5f30410928e325e8df617e6003605"
server_key_iv: "69e93f3cb5d65eb543660392aefc04d9"
server_key_version: "encryptionkey@0"
I20260928 16:01:54.401594 9657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "1c2d711bdf8f4fa6a073760d76d216d4"
format_stamp: "Formatted at 2026-09-28 16:01:54 on dist-test-slave-9cqd"
server_key: "3db5f30410928e325e8df617e6003605"
server_key_iv: "69e93f3cb5d65eb543660392aefc04d9"
server_key_version: "encryptionkey@0"
I20260928 16:01:54.406443 9657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:01:54.409756 9671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:54.411041 9657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:01:54.411214 9657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "1c2d711bdf8f4fa6a073760d76d216d4"
format_stamp: "Formatted at 2026-09-28 16:01:54 on dist-test-slave-9cqd"
server_key: "3db5f30410928e325e8df617e6003605"
server_key_iv: "69e93f3cb5d65eb543660392aefc04d9"
server_key_version: "encryptionkey@0"
I20260928 16:01:54.411419 9657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:54.425287 9657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:54.426000 9657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:54.426297 9657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:54.427170 9657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:54.428634 9657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:54.428755 9657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:54.428890 9657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:54.428977 9657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:54.463742 9657 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:45481
I20260928 16:01:54.463829 9783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:45481 every 8 connection(s)
I20260928 16:01:54.465106 9657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:01:54.469305 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 9657
I20260928 16:01:54.469483 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:01:54.470042 23031 external_mini_cluster.cc:1428] Setting key 179fd92e3ab8a41874a7dc3dcc2a1c2f
I20260928 16:01:54.473053 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35901
--builtin_ntp_servers=127.22.125.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:01:54.479628 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.467769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:56785 (local address 127.22.125.254:35901)
0928 16:01:54.467951 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:54.467957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:54.469727 (+ 1770us) server_negotiation.cc:407] Connection header received
0928 16:01:54.471869 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:54.471875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:54.471960 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:54.472091 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:54.473435 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.474679 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.476499 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.477265 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.477397 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.478680 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:54.478689 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:54.478696 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:54.478721 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:54.478764 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:54.478782 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:54.478870 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:54.479075 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 16:01:54.479196 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:01:54.480459 9786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.468404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35901 (local address 127.22.125.194:56785)
0928 16:01:54.469507 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:54.469585 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:54.471604 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:54.472365 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:54.472376 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:54.472481 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:54.473207 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.473229 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.474843 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.474849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.476362 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.476375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.477453 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.477458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.477643 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.478365 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:01:54.478368 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:54.478404 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:01:54.478474 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:54.478873 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:54.478881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:54.478887 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 16:01:54.479013 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 16:01:54.479281 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":289,"threads_started":1}
I20260928 16:01:54.481962 9784 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35901
I20260928 16:01:54.482254 9784 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:54.482982 9784 heartbeater.cc:511] Master 127.22.125.254:35901 requested a full tablet report, sending...
I20260928 16:01:54.484403 9466 ts_manager.cc:194] Registered new tserver with Master: 1c2d711bdf8f4fa6a073760d76d216d4 (127.22.125.194:45481)
I20260928 16:01:54.485638 9466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.194:56785
W20260928 16:01:54.799487 9787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:54.799748 9787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:54.799840 9787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:54.807209 9787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:54.807308 9787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:54.807471 9787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:01:54.817073 9787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:35901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:54.818346 9787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:54.820107 9787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:54.832952 9793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:54.832999 9792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:54.832996 9795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:54.833554 9787 server_base.cc:1034] running on GCE node
I20260928 16:01:54.834270 9787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:54.835239 9787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:54.836468 9787 hybrid_clock.cc:648] HybridClock initialized: now 1790611314836429 us; error 52 us; skew 500 ppm
I20260928 16:01:54.836800 9787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:01:54.839013 9787 webserver.cc:460] Webserver started at http://127.22.125.195:34315/ using document root <none> and password file <none>
I20260928 16:01:54.839682 9787 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:54.839818 9787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:54.840283 9787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:54.843055 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "90e4d0142d8a49e6bd4f5c4e44480548"
format_stamp: "Formatted at 2026-09-28 16:01:54 on dist-test-slave-9cqd"
server_key: "c32f42129f70707f0df0492755b3d7eb"
server_key_iv: "6c143f2b3d1c2c5460eb45fa59760a96"
server_key_version: "encryptionkey@0"
I20260928 16:01:54.843838 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "90e4d0142d8a49e6bd4f5c4e44480548"
format_stamp: "Formatted at 2026-09-28 16:01:54 on dist-test-slave-9cqd"
server_key: "c32f42129f70707f0df0492755b3d7eb"
server_key_iv: "6c143f2b3d1c2c5460eb45fa59760a96"
server_key_version: "encryptionkey@0"
I20260928 16:01:54.848832 9787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260928 16:01:54.852176 9801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:54.853492 9787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:54.853672 9787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "90e4d0142d8a49e6bd4f5c4e44480548"
format_stamp: "Formatted at 2026-09-28 16:01:54 on dist-test-slave-9cqd"
server_key: "c32f42129f70707f0df0492755b3d7eb"
server_key_iv: "6c143f2b3d1c2c5460eb45fa59760a96"
server_key_version: "encryptionkey@0"
I20260928 16:01:54.853895 9787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:54.882591 9787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:54.883436 9787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:54.883744 9787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:54.884717 9787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:54.886157 9787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:54.886277 9787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:54.886413 9787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:54.886500 9787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:54.917724 9787 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:35555
I20260928 16:01:54.917815 9913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:35555 every 8 connection(s)
I20260928 16:01:54.919034 9787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:01:54.919318 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 9787
I20260928 16:01:54.919492 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:01:54.920187 23031 external_mini_cluster.cc:1428] Setting key e9056838b55a5a5527da630d7f99fdc1
I20260928 16:01:54.931629 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.921786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:52757 (local address 127.22.125.254:35901)
0928 16:01:54.921972 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:54.921995 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:54.923608 (+ 1613us) server_negotiation.cc:407] Connection header received
0928 16:01:54.925358 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:54.925361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:54.925425 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:54.925507 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:54.926491 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.927196 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.929105 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.929701 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.929799 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.931037 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:54.931042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:54.931045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:54.931061 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:54.931088 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:54.931099 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:54.931163 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:54.931329 (+ 166us) server_negotiation.cc:299] Negotiation successful
0928 16:01:54.931403 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 16:01:54.932587 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.922420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35901 (local address 127.22.125.195:52757)
0928 16:01:54.923451 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:54.923494 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:54.925133 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:54.925700 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:54.925707 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:54.925788 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:54.926349 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.926369 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.927326 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.927329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.928935 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.928951 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.929881 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.929886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.930074 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.930763 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:01:54.930766 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:54.930806 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:01:54.930884 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:54.931161 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:54.931166 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:54.931170 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 16:01:54.931296 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 16:01:54.931556 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":360,"threads_started":1}
I20260928 16:01:54.933846 9914 heartbeater.cc:347] Connected to a master server at 127.22.125.254:35901
I20260928 16:01:54.934129 9914 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:54.935021 9914 heartbeater.cc:511] Master 127.22.125.254:35901 requested a full tablet report, sending...
I20260928 16:01:54.936385 9466 ts_manager.cc:194] Registered new tserver with Master: 90e4d0142d8a49e6bd4f5c4e44480548 (127.22.125.195:35555)
I20260928 16:01:54.937155 9466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.195:52757
I20260928 16:01:54.947073 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:01:54.956949 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.950981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.22.125.254:35901)
0928 16:01:54.951171 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:54.951179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:54.951599 (+ 420us) server_negotiation.cc:407] Connection header received
0928 16:01:54.951735 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:54.951739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:54.951807 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:54.951884 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:54.952409 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.953139 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.954438 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.954843 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.954936 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.956131 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:54.956137 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:54.956141 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:54.956157 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:54.956187 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:54.956200 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:54.956309 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:54.956658 (+ 349us) server_negotiation.cc:299] Negotiation successful
0928 16:01:54.956717 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:01:54.960659 9466 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:01:54.967308 9466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:01:54.989430 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.978947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:45481 (local address 127.0.0.1:44568)
0928 16:01:54.980214 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:54.980268 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:54.980486 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:54.982773 (+ 2287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:54.982778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:54.982806 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:54.983162 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.983174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.985019 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.985024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.986634 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.986647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.988547 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.988554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.988733 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.988767 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 16:01:54.988837 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 16:01:54.989053 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":25,"spinlock_wait_cycles":4992,"thread_start_us":706,"threads_started":1}
I20260928 16:01:54.989462 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.978791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.22.125.193:32883)
0928 16:01:54.979477 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:54.979503 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:54.980832 (+ 1329us) server_negotiation.cc:407] Connection header received
0928 16:01:54.981137 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:54.981145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:54.981395 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:54.981567 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:54.982617 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.984013 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.986642 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.987841 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.987998 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.988245 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:54.988923 (+ 678us) server_negotiation.cc:299] Negotiation successful
0928 16:01:54.989100 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":440,"threads_started":1}
I20260928 16:01:54.989787 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.979193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.22.125.194:45481)
0928 16:01:54.982258 (+ 3065us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:54.982264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:54.982285 (+ 21us) server_negotiation.cc:407] Connection header received
0928 16:01:54.982361 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:54.982365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:54.982509 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:54.982775 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:54.983345 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.984855 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.986817 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.988329 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.989087 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.989282 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:54.989354 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 16:01:54.989516 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2926,"thread_start_us":2825,"threads_started":1}
I20260928 16:01:54.990756 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.978523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58328)
0928 16:01:54.980718 (+ 2195us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:54.980736 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:54.980937 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:54.981914 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:54.981923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:54.981955 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:54.982398 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.982415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.984203 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.984208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.986460 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.986472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.988370 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.988376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.988614 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.988672 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 16:01:54.988867 (+ 195us) client_negotiation.cc:240] Negotiation successful
0928 16:01:54.990393 (+ 1526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1333,"mutex_wait_us":1455,"thread_start_us":1791,"threads_started":1}
I20260928 16:01:54.991130 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.979655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:35555 (local address 127.0.0.1:42186)
0928 16:01:54.980367 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:54.980383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:54.980486 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:54.982067 (+ 1581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:54.982071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:54.982090 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:54.982561 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.982571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.984194 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.984200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.988131 (+ 3931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:54.988147 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.990347 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.990354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:54.990511 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.990542 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 16:01:54.990633 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 16:01:54.990774 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":270,"thread_start_us":689,"threads_started":1}
I20260928 16:01:54.991885 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:54.980204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.22.125.195:35555)
0928 16:01:54.981094 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:54.981102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:54.981132 (+ 30us) server_negotiation.cc:407] Connection header received
0928 16:01:54.981217 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:54.981223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:54.981399 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:54.981571 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:54.982709 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.983963 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.988357 (+ 4394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:54.989802 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:54.990024 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:54.990214 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:54.991385 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0928 16:01:54.991551 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":571,"threads_started":1}
I20260928 16:01:54.993432 9719 tablet_service.cc:1467] Processing CreateTablet for tablet 793afb0d07e349cda292537a951f7965 (DEFAULT_TABLE table=test-table [id=f4a230cdf8744a5bb5a990af31a0b2ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:54.994271 9587 tablet_service.cc:1467] Processing CreateTablet for tablet 793afb0d07e349cda292537a951f7965 (DEFAULT_TABLE table=test-table [id=f4a230cdf8744a5bb5a990af31a0b2ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:54.994874 9719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793afb0d07e349cda292537a951f7965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:54.995144 9849 tablet_service.cc:1467] Processing CreateTablet for tablet 793afb0d07e349cda292537a951f7965 (DEFAULT_TABLE table=test-table [id=f4a230cdf8744a5bb5a990af31a0b2ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:54.995532 9587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793afb0d07e349cda292537a951f7965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:54.996233 9849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793afb0d07e349cda292537a951f7965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:55.007388 9932 tablet_bootstrap.cc:492] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Bootstrap starting.
I20260928 16:01:55.009474 9933 tablet_bootstrap.cc:492] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: Bootstrap starting.
I20260928 16:01:55.010228 9934 tablet_bootstrap.cc:492] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Bootstrap starting.
I20260928 16:01:55.011132 9932 tablet_bootstrap.cc:654] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:55.012455 9932 log.cc:824] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:55.013633 9934 tablet_bootstrap.cc:654] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:55.015056 9934 log.cc:824] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:55.015496 9933 tablet_bootstrap.cc:654] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:55.015726 9932 tablet_bootstrap.cc:492] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: No bootstrap required, opened a new log
I20260928 16:01:55.016134 9932 ts_tablet_manager.cc:1397] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260928 16:01:55.016819 9933 log.cc:824] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:55.018356 9934 tablet_bootstrap.cc:492] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: No bootstrap required, opened a new log
I20260928 16:01:55.018615 9934 ts_tablet_manager.cc:1397] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260928 16:01:55.019230 9933 tablet_bootstrap.cc:492] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: No bootstrap required, opened a new log
I20260928 16:01:55.019490 9933 ts_tablet_manager.cc:1397] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260928 16:01:55.026433 9932 raft_consensus.cc:348] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.027046 9932 raft_consensus.cc:374] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:55.027204 9932 raft_consensus.cc:729] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e4d0142d8a49e6bd4f5c4e44480548, State: Initialized, Role: FOLLOWER
I20260928 16:01:55.027892 9932 consensus_queue.cc:260] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.029704 9933 raft_consensus.cc:348] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.030247 9933 raft_consensus.cc:374] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:55.030414 9933 raft_consensus.cc:729] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98b200ac578247f883fa48fc36c6770c, State: Initialized, Role: FOLLOWER
I20260928 16:01:55.031057 9933 consensus_queue.cc:260] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.033066 9934 raft_consensus.cc:348] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.033672 9934 raft_consensus.cc:374] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:55.033862 9934 raft_consensus.cc:729] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2d711bdf8f4fa6a073760d76d216d4, State: Initialized, Role: FOLLOWER
I20260928 16:01:55.034698 9934 consensus_queue.cc:260] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.036024 9914 heartbeater.cc:503] Master 127.22.125.254:35901 was elected leader, sending a full tablet report...
I20260928 16:01:55.036721 9932 ts_tablet_manager.cc:1428] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Time spent starting tablet: real 0.020s user 0.011s sys 0.009s
I20260928 16:01:55.037585 9784 heartbeater.cc:503] Master 127.22.125.254:35901 was elected leader, sending a full tablet report...
I20260928 16:01:55.038967 9933 ts_tablet_manager.cc:1428] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: Time spent starting tablet: real 0.019s user 0.019s sys 0.000s
I20260928 16:01:55.039256 9653 heartbeater.cc:503] Master 127.22.125.254:35901 was elected leader, sending a full tablet report...
I20260928 16:01:55.039695 9934 ts_tablet_manager.cc:1428] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Time spent starting tablet: real 0.021s user 0.016s sys 0.000s
I20260928 16:01:55.056367 9939 raft_consensus.cc:482] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:55.056641 9939 raft_consensus.cc:504] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.058388 9939 leader_election.cc:288] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c2d711bdf8f4fa6a073760d76d216d4 (127.22.125.194:45481), 90e4d0142d8a49e6bd4f5c4e44480548 (127.22.125.195:35555)
I20260928 16:01:55.065572 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.058934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:40649 (local address 127.22.125.195:35555)
0928 16:01:55.059141 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:55.059150 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:55.059869 (+ 719us) server_negotiation.cc:407] Connection header received
0928 16:01:55.060086 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:55.060091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:55.060165 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:55.060248 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:55.061148 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.061881 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.063706 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.064602 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.064786 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.064863 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:55.065197 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 16:01:55.065321 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 16:01:55.065580 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.058737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:54165 (local address 127.22.125.194:45481)
0928 16:01:55.058939 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:55.058946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:55.059699 (+ 753us) server_negotiation.cc:407] Connection header received
0928 16:01:55.059955 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:55.059959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:55.060080 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:55.060145 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:55.061072 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.061772 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.063759 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.064589 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.064738 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.064828 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:55.065195 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 16:01:55.065324 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:01:55.065692 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.058735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:45481 (local address 127.22.125.193:54165)
0928 16:01:55.059579 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.059621 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:55.059766 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:55.060362 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:55.060367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:55.060393 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:55.060908 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:55.060926 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.061913 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.061922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:55.063553 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:55.063565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.064777 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.064783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:55.064971 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.065064 (+ 93us) client_negotiation.cc:769] Sending connection context
0928 16:01:55.065168 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 16:01:55.065321 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":181,"thread_start_us":434,"threads_started":1}
I20260928 16:01:55.066021 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.058844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:35555 (local address 127.22.125.193:40649)
0928 16:01:55.059760 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.059777 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:55.059924 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:55.060291 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:55.060304 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:55.060339 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:55.060972 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:55.060984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.062020 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.062024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:55.063526 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:55.063538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.064811 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.064817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:55.065014 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.065067 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 16:01:55.065168 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 16:01:55.065323 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":120,"thread_start_us":584,"threads_started":1}
I20260928 16:01:55.066844 9739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793afb0d07e349cda292537a951f7965" candidate_uuid: "98b200ac578247f883fa48fc36c6770c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" is_pre_election: true
I20260928 16:01:55.066871 9869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793afb0d07e349cda292537a951f7965" candidate_uuid: "98b200ac578247f883fa48fc36c6770c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" is_pre_election: true
I20260928 16:01:55.067339 9869 raft_consensus.cc:2454] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98b200ac578247f883fa48fc36c6770c in term 0.
I20260928 16:01:55.067344 9739 raft_consensus.cc:2454] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98b200ac578247f883fa48fc36c6770c in term 0.
I20260928 16:01:55.067977 9540 leader_election.cc:302] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2d711bdf8f4fa6a073760d76d216d4, 98b200ac578247f883fa48fc36c6770c; no voters:
I20260928 16:01:55.068445 9939 raft_consensus.cc:2781] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:01:55.068598 9939 raft_consensus.cc:482] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:01:55.068706 9939 raft_consensus.cc:3037] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:55.070818 9939 raft_consensus.cc:504] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.071434 9939 leader_election.cc:288] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [CANDIDATE]: Term 1 election: Requested vote from peers 1c2d711bdf8f4fa6a073760d76d216d4 (127.22.125.194:45481), 90e4d0142d8a49e6bd4f5c4e44480548 (127.22.125.195:35555)
I20260928 16:01:55.071748 9739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793afb0d07e349cda292537a951f7965" candidate_uuid: "98b200ac578247f883fa48fc36c6770c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c2d711bdf8f4fa6a073760d76d216d4"
I20260928 16:01:55.071930 9739 raft_consensus.cc:3037] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:55.071931 9869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793afb0d07e349cda292537a951f7965" candidate_uuid: "98b200ac578247f883fa48fc36c6770c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e4d0142d8a49e6bd4f5c4e44480548"
I20260928 16:01:55.072141 9869 raft_consensus.cc:3037] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:55.074137 9739 raft_consensus.cc:2454] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98b200ac578247f883fa48fc36c6770c in term 1.
I20260928 16:01:55.074337 9869 raft_consensus.cc:2454] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98b200ac578247f883fa48fc36c6770c in term 1.
I20260928 16:01:55.074532 9540 leader_election.cc:302] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2d711bdf8f4fa6a073760d76d216d4, 98b200ac578247f883fa48fc36c6770c; no voters:
I20260928 16:01:55.074821 9939 raft_consensus.cc:2781] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:55.075227 9939 raft_consensus.cc:686] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 1 LEADER]: Becoming Leader. State: Replica: 98b200ac578247f883fa48fc36c6770c, State: Running, Role: LEADER
I20260928 16:01:55.075670 9939 consensus_queue.cc:237] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } }
I20260928 16:01:55.079819 9466 catalog_manager.cc:5495] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c reported cstate change: term changed from 0 to 1, leader changed from <none> to 98b200ac578247f883fa48fc36c6770c (127.22.125.193). New cstate: current_term: 1 leader_uuid: "98b200ac578247f883fa48fc36c6770c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b200ac578247f883fa48fc36c6770c" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 32883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 } health_report { overall_health: UNKNOWN } } }
I20260928 16:01:55.116760 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.110906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.22.125.193:32883)
0928 16:01:55.111113 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:55.111119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:55.111220 (+ 101us) server_negotiation.cc:407] Connection header received
0928 16:01:55.111476 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:55.111480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:55.111567 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:55.111720 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:01:55.112151 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.112837 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.114296 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.114809 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.114942 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.115556 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:01:55.116168 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:01:55.116351 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:55.116483 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 16:01:55.116525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:01:55.124631 9869 raft_consensus.cc:1261] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 1 FOLLOWER]: Refusing update from remote peer 98b200ac578247f883fa48fc36c6770c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:55.124662 9739 raft_consensus.cc:1261] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 1 FOLLOWER]: Refusing update from remote peer 98b200ac578247f883fa48fc36c6770c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:55.125582 9943 consensus_queue.cc:1035] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c2d711bdf8f4fa6a073760d76d216d4" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 45481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:55.125941 9939 consensus_queue.cc:1035] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e4d0142d8a49e6bd4f5c4e44480548" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 35555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:55.136960 9946 mvcc.cc:205] Tried to move back new op lower bound from 7334343946741129216 to 7334343946554068992. Current Snapshot: MvccSnapshot[applied={T|T < 7334343946741129216}]
I20260928 16:01:55.151723 9465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:55.152118 9465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:55.155922 9465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71f4545483e34cb4a789a43dcbc58dbb: Sending DeleteTablet for 3 replicas of tablet 793afb0d07e349cda292537a951f7965
I20260928 16:01:55.157025 9587 tablet_service.cc:1514] Processing DeleteTablet for tablet 793afb0d07e349cda292537a951f7965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:55 UTC) from {username='slave'} at 127.0.0.1:58328
I20260928 16:01:55.157042 9719 tablet_service.cc:1514] Processing DeleteTablet for tablet 793afb0d07e349cda292537a951f7965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:55 UTC) from {username='slave'} at 127.0.0.1:44568
I20260928 16:01:55.157063 9849 tablet_service.cc:1514] Processing DeleteTablet for tablet 793afb0d07e349cda292537a951f7965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:55 UTC) from {username='slave'} at 127.0.0.1:42186
I20260928 16:01:55.157979 9956 tablet_replica.cc:330] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: stopping tablet replica
I20260928 16:01:55.158669 9956 raft_consensus.cc:2229] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:55.159344 9956 raft_consensus.cc:2258] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:55.161629 9956 ts_tablet_manager.cc:1905] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:55.163134 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 9525
I20260928 16:01:55.169329 9956 ts_tablet_manager.cc:1918] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:55.169603 9956 log.cc:1196] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/793afb0d07e349cda292537a951f7965
I20260928 16:01:55.169708 9957 tablet_replica.cc:330] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: stopping tablet replica
I20260928 16:01:55.170132 9957 raft_consensus.cc:2229] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:55.170500 9956 ts_tablet_manager.cc:1939] T 793afb0d07e349cda292537a951f7965 P 1c2d711bdf8f4fa6a073760d76d216d4: Deleting consensus metadata
I20260928 16:01:55.170589 9957 raft_consensus.cc:2258] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:55.172730 9450 catalog_manager.cc:4841] TS 1c2d711bdf8f4fa6a073760d76d216d4 (127.22.125.194:45481): tablet 793afb0d07e349cda292537a951f7965 (table test-table [id=f4a230cdf8744a5bb5a990af31a0b2ca]) successfully deleted
I20260928 16:01:55.174360 9957 ts_tablet_manager.cc:1905] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:55.177942 9958 tablet_replica.cc:330] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: stopping tablet replica
I20260928 16:01:55.178319 9958 raft_consensus.cc:2229] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:01:55.179075 9958 raft_consensus.cc:2258] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:55.181067 9957 ts_tablet_manager.cc:1918] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:55.181216 9957 log.cc:1196] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/793afb0d07e349cda292537a951f7965
I20260928 16:01:55.181221 9958 ts_tablet_manager.cc:1905] T 793afb0d07e349cda292537a951f7965 P 98b200ac578247f883fa48fc36c6770c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:55.181735 9957 ts_tablet_manager.cc:1939] T 793afb0d07e349cda292537a951f7965 P 90e4d0142d8a49e6bd4f5c4e44480548: Deleting consensus metadata
I20260928 16:01:55.183401 9452 catalog_manager.cc:4841] TS 90e4d0142d8a49e6bd4f5c4e44480548 (127.22.125.195:35555): tablet 793afb0d07e349cda292537a951f7965 (table test-table [id=f4a230cdf8744a5bb5a990af31a0b2ca]) successfully deleted
I20260928 16:01:55.184719 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.164267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.22.125.193:32883)
0928 16:01:55.164430 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:55.164436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:55.175675 (+ 11239us) server_negotiation.cc:407] Connection header received
0928 16:01:55.175917 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:55.175923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:55.176024 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:55.176182 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:55.177092 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.178447 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.180542 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.181181 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.181301 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.182935 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:55.182947 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:55.182970 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:55.183725 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:55.183787 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:55.183812 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:55.183903 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:55.184241 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 16:01:55.184353 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 16:01:55.186211 9648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:55.283059 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 9525
I20260928 16:01:55.301334 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 9657
W20260928 16:01:55.301501 9451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.125.193:32883 (error 108)
I20260928 16:01:55.303345 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.302921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58348)
0928 16:01:55.303102 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.303199 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 16:01:55.303565 9451 catalog_manager.cc:4584] TS 98b200ac578247f883fa48fc36c6770c (127.22.125.193:32883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 793afb0d07e349cda292537a951f7965: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
I20260928 16:01:55.308525 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.302146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.22.125.194:45481)
0928 16:01:55.302339 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:55.302345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:55.302478 (+ 133us) server_negotiation.cc:407] Connection header received
0928 16:01:55.302680 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:55.302686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:55.302800 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:55.302914 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:55.303333 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.304269 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.305612 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.306022 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.306177 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.307171 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:55.307180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:55.307193 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:55.307741 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:55.307787 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:55.307807 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:55.307944 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:55.308117 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 16:01:55.308218 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:01:55.309175 9779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:55.357213 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.356649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58356)
0928 16:01:55.356906 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.357017 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260928 16:01:55.358282 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.357918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58364)
0928 16:01:55.358086 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.358155 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 16:01:55.399899 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 9657
I20260928 16:01:55.416834 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 9787
I20260928 16:01:55.418656 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.418129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58368)
0928 16:01:55.418349 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.418448 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260928 16:01:55.419608 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.419290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58382)
0928 16:01:55.419436 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.419500 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 16:01:55.423676 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.417563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.22.125.195:35555)
0928 16:01:55.417756 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:55.417761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:55.417784 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:01:55.417955 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:55.417963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:55.418084 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:55.418202 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:55.418670 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.419636 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.421017 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:55.421411 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:55.421508 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:55.422483 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:55.422490 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:55.422505 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:01:55.423098 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:55.423142 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:01:55.423160 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:55.423252 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:55.423357 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 16:01:55.423438 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:01:55.424368 9909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:55.500198 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.499705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58396)
0928 16:01:55.499909 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.500004 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 16:01:55.501401 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:55.501006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:32883 (local address 127.0.0.1:58406)
0928 16:01:55.501215 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:55.501285 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.193:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260928 16:01:55.514772 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 9787
I20260928 16:01:55.531757 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 9435
I20260928 16:01:55.532939 9496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:55.642177 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 9435
2026-09-28T16:01:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2679 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:01:55 dist-test-slave-9cqd krb5kdc[9967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:01:55 dist-test-slave-9cqd krb5kdc[9967](info): set up 2 sockets
Sep 28 16:01:55 dist-test-slave-9cqd krb5kdc[9967](info): commencing operation
krb5kdc: starting...
W20260928 16:01:56.733893 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:01:56 dist-test-slave-9cqd krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:01:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:01:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:01:56.969090 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42761
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:44367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:42761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:57.293397 9980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:57.293663 9980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:57.293779 9980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:57.300915 9980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:01:57.301009 9980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:57.301124 9980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:57.301221 9980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:01:57.301301 9980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:01:57.310699 9980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:44367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:42761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:42761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:57.311925 9980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:57.313241 9980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:57.324910 9985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:57.324952 9986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:57.324999 9988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:57.325750 9980 server_base.cc:1034] running on GCE node
I20260928 16:01:57.326565 9980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:57.327903 9980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:57.329145 9980 hybrid_clock.cc:648] HybridClock initialized: now 1790611317329053 us; error 77 us; skew 500 ppm
I20260928 16:01:57.329486 9980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:57 dist-test-slave-9cqd krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:57.340590 9980 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:01:57.342152 9980 webserver.cc:460] Webserver started at http://127.22.125.254:43797/ using document root <none> and password file <none>
I20260928 16:01:57.342864 9980 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:57.343017 9980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:57.343324 9980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:57.346338 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "f6c55bdf5e154a828392495eeefad442"
format_stamp: "Formatted at 2026-09-28 16:01:57 on dist-test-slave-9cqd"
server_key: "f365c3a0c80a6ef0e87b9d20addf2f32"
server_key_iv: "2697826e680ac61938eb7cf595ef265a"
server_key_version: "encryptionkey@0"
I20260928 16:01:57.347078 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "f6c55bdf5e154a828392495eeefad442"
format_stamp: "Formatted at 2026-09-28 16:01:57 on dist-test-slave-9cqd"
server_key: "f365c3a0c80a6ef0e87b9d20addf2f32"
server_key_iv: "2697826e680ac61938eb7cf595ef265a"
server_key_version: "encryptionkey@0"
I20260928 16:01:57.352113 9980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260928 16:01:57.355365 9995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:57.356688 9980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 16:01:57.356878 9980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "f6c55bdf5e154a828392495eeefad442"
format_stamp: "Formatted at 2026-09-28 16:01:57 on dist-test-slave-9cqd"
server_key: "f365c3a0c80a6ef0e87b9d20addf2f32"
server_key_iv: "2697826e680ac61938eb7cf595ef265a"
server_key_version: "encryptionkey@0"
I20260928 16:01:57.357075 9980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:57.380028 9980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:57.384399 9980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:57.384687 9980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:57.406309 9980 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:42761
I20260928 16:01:57.406333 10046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:42761 every 8 connection(s)
I20260928 16:01:57.407660 9980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:01:57.411389 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 9980
I20260928 16:01:57.411433 10047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:57.411567 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:01:57.412137 23031 external_mini_cluster.cc:1428] Setting key d94fe98ae22044dac251b70a87f50518
I20260928 16:01:57.423593 10047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442: Bootstrap starting.
Sep 28 16:01:57 dist-test-slave-9cqd krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:57.426522 10047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:57.427534 10047 log.cc:824] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:57.430208 10047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442: No bootstrap required, opened a new log
I20260928 16:01:57.433244 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:57.414017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.22.125.254:42761)
0928 16:01:57.414810 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:57.414827 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:57.414870 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:01:57.416459 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:57.416480 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:57.416848 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:57.417239 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:57.418696 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:57.419936 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:57.421252 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:57.421946 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:57.422165 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:57.426151 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:57.426159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:57.426173 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:57.426210 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:57.429260 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:57.430004 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:57.430016 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:57.430021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:57.430173 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:57.430568 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:57.430573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:57.430577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:57.431048 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:57.431365 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:57.431850 (+ 485us) server_negotiation.cc:299] Negotiation successful
0928 16:01:57.432297 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":356,"threads_started":1}
I20260928 16:01:57.437824 10047 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55bdf5e154a828392495eeefad442" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42761 } }
I20260928 16:01:57.438169 10047 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:57.438246 10047 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c55bdf5e154a828392495eeefad442, State: Initialized, Role: FOLLOWER
I20260928 16:01:57.438683 10047 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55bdf5e154a828392495eeefad442" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42761 } }
I20260928 16:01:57.438882 10047 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:01:57.438970 10047 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:01:57.439077 10047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:57.441457 10047 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55bdf5e154a828392495eeefad442" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42761 } }
I20260928 16:01:57.441876 10047 leader_election.cc:302] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6c55bdf5e154a828392495eeefad442; no voters:
I20260928 16:01:57.442438 10047 leader_election.cc:288] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:01:57.442534 10052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:57.443270 10052 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [term 1 LEADER]: Becoming Leader. State: Replica: f6c55bdf5e154a828392495eeefad442, State: Running, Role: LEADER
I20260928 16:01:57.443861 10052 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55bdf5e154a828392495eeefad442" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42761 } }
I20260928 16:01:57.444736 10047 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:01:57.447139 10053 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6c55bdf5e154a828392495eeefad442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55bdf5e154a828392495eeefad442" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42761 } } }
I20260928 16:01:57.447177 10054 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6c55bdf5e154a828392495eeefad442. Latest consensus state: current_term: 1 leader_uuid: "f6c55bdf5e154a828392495eeefad442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55bdf5e154a828392495eeefad442" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 42761 } } }
I20260928 16:01:57.447697 10053 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:57.447728 10054 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442 [sys.catalog]: This master's current role is: LEADER
I20260928 16:01:57.449038 10061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:01:57.455356 10061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:01:57.463565 10061 catalog_manager.cc:1347] Generated new cluster ID: 01f6e3c578784da1b199c6bb77ce20f3
I20260928 16:01:57.463673 10061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:01:57.504184 10061 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:01:57.505533 10061 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:01:57.516129 10061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442: Generated new TSK 0
I20260928 16:01:57.517148 10061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.125.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
Entry for principal kudu/127.22.125.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab.
I20260928 16:01:57.591213 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42761
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:44367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:57.926270 10073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:57.926560 10073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:57.926728 10073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:57.933712 10073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:57.933852 10073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:57.934017 10073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:01:57.943444 10073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:44367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.193.keytab
--principal=kudu/127.22.125.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:57.944717 10073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:57.946151 10073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:57.959111 10078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:57.959309 10081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:57.959074 10079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:57.960228 10073 server_base.cc:1034] running on GCE node
I20260928 16:01:57.960855 10073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:57.961758 10073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:57.962965 10073 hybrid_clock.cc:648] HybridClock initialized: now 1790611317962919 us; error 39 us; skew 500 ppm
I20260928 16:01:57.963248 10073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:57 dist-test-slave-9cqd krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:57.966166 10073 init.cc:377] Logged in from keytab as kudu/127.22.125.193@KRBTEST.COM (short username kudu)
I20260928 16:01:57.967418 10073 webserver.cc:460] Webserver started at http://127.22.125.193:41669/ using document root <none> and password file <none>
I20260928 16:01:57.967986 10073 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:57.968166 10073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:57.968433 10073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:57.971056 10073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "5dbe0aece9c94e7289be75920be49ac4"
format_stamp: "Formatted at 2026-09-28 16:01:57 on dist-test-slave-9cqd"
server_key: "0289d937a44f7d7a3d31a9581d614808"
server_key_iv: "9cf53f7e11198e03b6af14e40adaf32d"
server_key_version: "encryptionkey@0"
I20260928 16:01:57.971808 10073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "5dbe0aece9c94e7289be75920be49ac4"
format_stamp: "Formatted at 2026-09-28 16:01:57 on dist-test-slave-9cqd"
server_key: "0289d937a44f7d7a3d31a9581d614808"
server_key_iv: "9cf53f7e11198e03b6af14e40adaf32d"
server_key_version: "encryptionkey@0"
I20260928 16:01:57.976788 10073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260928 16:01:57.980105 10088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:57.981352 10073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:01:57.981524 10073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "5dbe0aece9c94e7289be75920be49ac4"
format_stamp: "Formatted at 2026-09-28 16:01:57 on dist-test-slave-9cqd"
server_key: "0289d937a44f7d7a3d31a9581d614808"
server_key_iv: "9cf53f7e11198e03b6af14e40adaf32d"
server_key_version: "encryptionkey@0"
I20260928 16:01:57.981719 10073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:58.016093 10073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:58.019968 10073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:58.020311 10073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:58.021239 10073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:58.022497 10073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:58.022603 10073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:58.022753 10073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:58.022825 10073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:58.055295 10073 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:39869
I20260928 16:01:58.055347 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:39869 every 8 connection(s)
I20260928 16:01:58.056676 10073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:01:58.057305 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 10073
I20260928 16:01:58.057500 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:01:58.058143 23031 external_mini_cluster.cc:1428] Setting key 28a3f31d8e655750171b8372374b6222
Sep 28 16:01:58 dist-test-slave-9cqd krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611317, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.193@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:58.078449 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:58.059664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:40909 (local address 127.22.125.254:42761)
0928 16:01:58.060185 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:58.060189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:58.061430 (+ 1241us) server_negotiation.cc:407] Connection header received
0928 16:01:58.063251 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:58.063258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:58.063349 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:58.063468 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:58.065005 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.066220 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.067739 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.068434 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.068539 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:58.072826 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:58.072831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:58.072834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:58.072865 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:58.075468 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:58.076211 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:58.076223 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:58.076227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:58.076342 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:58.076778 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:58.076782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:58.076787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:58.077038 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:58.077205 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:58.077853 (+ 648us) server_negotiation.cc:299] Negotiation successful
0928 16:01:58.078081 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":235,"threads_started":1}
I20260928 16:01:58.079363 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:58.060423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42761 (local address 127.22.125.193:40909)
0928 16:01:58.061270 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:58.061323 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:58.063000 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:58.063662 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:58.063670 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:58.064256 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:58.064824 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:58.064842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.066377 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.066381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:58.067594 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:58.067606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.068706 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.068716 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:58.069046 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:58.069765 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:58.069814 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:58.072619 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:58.075613 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:58.075621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:58.075635 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:58.076033 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:58.076452 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:58.076457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:58.076461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:58.076644 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:58.077171 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:58.077179 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:58.077498 (+ 319us) client_negotiation.cc:769] Sending connection context
0928 16:01:58.077759 (+ 261us) client_negotiation.cc:240] Negotiation successful
0928 16:01:58.078249 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":406,"threads_started":1}
I20260928 16:01:58.081072 10201 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42761
I20260928 16:01:58.081387 10201 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:58.082082 10201 heartbeater.cc:511] Master 127.22.125.254:42761 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.125.194@KRBTEST.COM; defaulting to no policy
I20260928 16:01:58.084431 10012 ts_manager.cc:194] Registered new tserver with Master: 5dbe0aece9c94e7289be75920be49ac4 (127.22.125.193:39869)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.194@KRBTEST.COM" created.
I20260928 16:01:58.086126 10012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.193@KRBTEST.COM'} at 127.22.125.193:40909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
Entry for principal kudu/127.22.125.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab.
I20260928 16:01:58.113565 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42761
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:44367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:58.435500 10207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:58.435750 10207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:58.435850 10207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:58.442845 10207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:58.442953 10207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:58.443058 10207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:01:58.452636 10207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:44367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.194.keytab
--principal=kudu/127.22.125.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:58.453882 10207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:58.455204 10207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:58.467633 10213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:58.467589 10215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:58.467670 10212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:58.468400 10207 server_base.cc:1034] running on GCE node
I20260928 16:01:58.469014 10207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:58.469918 10207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:58.471143 10207 hybrid_clock.cc:648] HybridClock initialized: now 1790611318471111 us; error 51 us; skew 500 ppm
I20260928 16:01:58.471443 10207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:58 dist-test-slave-9cqd krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611318, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:58.474622 10207 init.cc:377] Logged in from keytab as kudu/127.22.125.194@KRBTEST.COM (short username kudu)
I20260928 16:01:58.476197 10207 webserver.cc:460] Webserver started at http://127.22.125.194:35017/ using document root <none> and password file <none>
I20260928 16:01:58.476850 10207 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:58.476994 10207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:58.477311 10207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:58.480127 10207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "db8be1376c5346d59125b6fa7b15e82d"
format_stamp: "Formatted at 2026-09-28 16:01:58 on dist-test-slave-9cqd"
server_key: "cdb770fc8aaa788b77fafb299724f114"
server_key_iv: "2debef3e7fbee7d0ff424a949025fd4e"
server_key_version: "encryptionkey@0"
I20260928 16:01:58.480897 10207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "db8be1376c5346d59125b6fa7b15e82d"
format_stamp: "Formatted at 2026-09-28 16:01:58 on dist-test-slave-9cqd"
server_key: "cdb770fc8aaa788b77fafb299724f114"
server_key_iv: "2debef3e7fbee7d0ff424a949025fd4e"
server_key_version: "encryptionkey@0"
I20260928 16:01:58.485975 10207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260928 16:01:58.489389 10222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:58.490628 10207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:01:58.490814 10207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "db8be1376c5346d59125b6fa7b15e82d"
format_stamp: "Formatted at 2026-09-28 16:01:58 on dist-test-slave-9cqd"
server_key: "cdb770fc8aaa788b77fafb299724f114"
server_key_iv: "2debef3e7fbee7d0ff424a949025fd4e"
server_key_version: "encryptionkey@0"
I20260928 16:01:58.491009 10207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:58.512951 10207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:58.517400 10207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:58.517690 10207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:58.518639 10207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:58.519937 10207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:58.520103 10207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:58.520255 10207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:58.520339 10207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:58.555053 10207 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:37819
I20260928 16:01:58.555099 10334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:37819 every 8 connection(s)
I20260928 16:01:58.556478 10207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:01:58.566593 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 10207
I20260928 16:01:58.566783 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:01:58.567229 23031 external_mini_cluster.cc:1428] Setting key e79d5ad6a08052a15dd0d103bd0edb3e
Sep 28 16:01:58 dist-test-slave-9cqd krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611318, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.194@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:58.577051 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:58.558965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:52347 (local address 127.22.125.254:42761)
0928 16:01:58.559134 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:58.559139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:58.560412 (+ 1273us) server_negotiation.cc:407] Connection header received
0928 16:01:58.562320 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:58.562325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:58.562419 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:58.562528 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:58.563982 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.564736 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.566344 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.566877 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.567018 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:58.571072 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:58.571077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:58.571080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:58.571111 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:58.573821 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:58.574707 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:58.574711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:58.574713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:58.574852 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:58.575318 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:58.575321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:58.575323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:58.575646 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:58.575763 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:58.576613 (+ 850us) server_negotiation.cc:299] Negotiation successful
0928 16:01:58.576785 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 16:01:58.578629 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:58.559406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42761 (local address 127.22.125.194:52347)
0928 16:01:58.560254 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:58.560298 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:58.562058 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:58.562756 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:58.562764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:58.563247 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:58.563806 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:58.563829 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.564914 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.564918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:58.566144 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:58.566156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:58.567120 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:58.567127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:58.567389 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:58.568125 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:58.568147 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:58.570847 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:58.573952 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:58.573963 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:58.573980 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:58.574587 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:58.574958 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:58.574966 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:58.574972 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:58.575212 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:58.575767 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:58.575778 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:58.576230 (+ 452us) client_negotiation.cc:769] Sending connection context
0928 16:01:58.576533 (+ 303us) client_negotiation.cc:240] Negotiation successful
0928 16:01:58.577099 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":259,"threads_started":1}
I20260928 16:01:58.580286 10335 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42761
I20260928 16:01:58.580601 10335 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:58.581311 10335 heartbeater.cc:511] Master 127.22.125.254:42761 requested a full tablet report, sending...
I20260928 16:01:58.582774 10012 ts_manager.cc:194] Registered new tserver with Master: db8be1376c5346d59125b6fa7b15e82d (127.22.125.194:37819)
I20260928 16:01:58.584100 10012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.194@KRBTEST.COM'} at 127.22.125.194:52347
WARNING: no policy specified for kudu/127.22.125.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
Entry for principal kudu/127.22.125.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab.
I20260928 16:01:58.624651 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42761
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:44367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:01:58.943032 10340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:01:58.943289 10340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:01:58.943377 10340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:01:58.950527 10340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:01:58.950624 10340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:01:58.950753 10340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:01:58.960389 10340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:44367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.195.keytab
--principal=kudu/127.22.125.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:42761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:01:58.961635 10340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:01:58.962867 10340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:01:58.975481 10346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:58.975446 10348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:01:58.975693 10345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:01:58.976432 10340 server_base.cc:1034] running on GCE node
I20260928 16:01:58.977072 10340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:01:58.977949 10340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:01:58.979144 10340 hybrid_clock.cc:648] HybridClock initialized: now 1790611318979101 us; error 44 us; skew 500 ppm
I20260928 16:01:58.979530 10340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:01:58 dist-test-slave-9cqd krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611318, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:01:58.982594 10340 init.cc:377] Logged in from keytab as kudu/127.22.125.195@KRBTEST.COM (short username kudu)
I20260928 16:01:58.983872 10340 webserver.cc:460] Webserver started at http://127.22.125.195:43875/ using document root <none> and password file <none>
I20260928 16:01:58.984567 10340 fs_manager.cc:359] Metadata directory not provided
I20260928 16:01:58.984663 10340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:01:58.984972 10340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:01:58.987860 10340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "df6cc44e3ef24b868a76fc18238d558d"
format_stamp: "Formatted at 2026-09-28 16:01:58 on dist-test-slave-9cqd"
server_key: "23fcfb9203fa9a02276276da1e80f99e"
server_key_iv: "d7c6edfb35058c059b510f4bcbba5659"
server_key_version: "encryptionkey@0"
I20260928 16:01:58.988613 10340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "df6cc44e3ef24b868a76fc18238d558d"
format_stamp: "Formatted at 2026-09-28 16:01:58 on dist-test-slave-9cqd"
server_key: "23fcfb9203fa9a02276276da1e80f99e"
server_key_iv: "d7c6edfb35058c059b510f4bcbba5659"
server_key_version: "encryptionkey@0"
I20260928 16:01:58.993387 10340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260928 16:01:58.996930 10355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:58.998170 10340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:01:58.998348 10340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "df6cc44e3ef24b868a76fc18238d558d"
format_stamp: "Formatted at 2026-09-28 16:01:58 on dist-test-slave-9cqd"
server_key: "23fcfb9203fa9a02276276da1e80f99e"
server_key_iv: "d7c6edfb35058c059b510f4bcbba5659"
server_key_version: "encryptionkey@0"
I20260928 16:01:58.998548 10340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:01:59.013824 10340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:01:59.018102 10340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:01:59.018376 10340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:01:59.019277 10340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:01:59.020557 10340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:01:59.020656 10340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:59.020807 10340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:01:59.020879 10340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:01:59.053493 10340 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:36341
I20260928 16:01:59.053548 10467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:36341 every 8 connection(s)
I20260928 16:01:59.054811 10340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:01:59.057938 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 10340
I20260928 16:01:59.058128 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:01:59.058678 23031 external_mini_cluster.cc:1428] Setting key 09d6d1b829d0b0280d485cf034aad3b4
Sep 28 16:01:59 dist-test-slave-9cqd krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611318, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.195@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:01:59.075874 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.057713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:42385 (local address 127.22.125.254:42761)
0928 16:01:59.057934 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.057941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.059404 (+ 1463us) server_negotiation.cc:407] Connection header received
0928 16:01:59.061785 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.061791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.061885 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.061993 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:59.063642 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.064462 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.065972 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.066375 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.066510 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.069816 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:59.069821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:59.069825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:59.069857 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:59.072690 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.073564 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.073574 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.073576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.073715 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.074222 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.074227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.074230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.074500 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:59.074657 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.075316 (+ 659us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.075526 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 16:01:59.076885 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.058342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:42761 (local address 127.22.125.195:42385)
0928 16:01:59.059216 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:59.059273 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:59.061503 (+ 2230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:59.062229 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:59.062237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:59.062920 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:01:59.063479 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.063495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.064612 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.064617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.065831 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.065843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.066601 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.066609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.066852 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.067512 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:01:59.067533 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:01:59.069618 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:01:59.072834 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:59.072842 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:59.072856 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:59.073431 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:59.073855 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:01:59.073859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:01:59.073862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:01:59.074097 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:01:59.074631 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:01:59.074639 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:01:59.074971 (+ 332us) client_negotiation.cc:769] Sending connection context
0928 16:01:59.075235 (+ 264us) client_negotiation.cc:240] Negotiation successful
0928 16:01:59.075712 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":361,"threads_started":1}
I20260928 16:01:59.078145 10468 heartbeater.cc:347] Connected to a master server at 127.22.125.254:42761
I20260928 16:01:59.078404 10468 heartbeater.cc:464] Registering TS with master...
I20260928 16:01:59.079071 10468 heartbeater.cc:511] Master 127.22.125.254:42761 requested a full tablet report, sending...
I20260928 16:01:59.080467 10012 ts_manager.cc:194] Registered new tserver with Master: df6cc44e3ef24b868a76fc18238d558d (127.22.125.195:36341)
I20260928 16:01:59.081245 10012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.125.195@KRBTEST.COM'} at 127.22.125.195:42385
I20260928 16:01:59.086154 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:01:59.088943 10201 heartbeater.cc:503] Master 127.22.125.254:42761 was elected leader, sending a full tablet report...
I20260928 16:01:59.102294 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.090413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48710 (local address 127.22.125.254:42761)
0928 16:01:59.090572 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.090576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.091046 (+ 470us) server_negotiation.cc:407] Connection header received
0928 16:01:59.091148 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.091152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.091209 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.091289 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:59.092732 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.093434 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.094739 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.095154 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.095249 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.097590 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:59.097594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:59.097597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:59.097626 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:59.099917 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.100521 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.100525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.100527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.100670 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.101012 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.101015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.101017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.101293 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:59.101500 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.101798 (+ 298us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.102010 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 16:01:59.105756 10012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:01:59.112499 10012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:01:59.137514 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.124486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.22.125.193:39869)
0928 16:01:59.125383 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.125396 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.126044 (+ 648us) server_negotiation.cc:407] Connection header received
0928 16:01:59.126456 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.126466 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.126725 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.127082 (+ 357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:59.128238 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.130185 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.133591 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.135525 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.135728 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.135961 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.136824 (+ 863us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.137058 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":627,"threads_started":1}
I20260928 16:01:59.138592 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.124274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39869 (local address 127.0.0.1:34732)
0928 16:01:59.125917 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:59.125952 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:59.126207 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:59.127449 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:59.127459 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:59.127498 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:59.127993 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.128008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.130391 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.130399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.133383 (+ 2984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.133402 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.136207 (+ 2805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.136215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.136499 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.136571 (+ 72us) client_negotiation.cc:769] Sending connection context
0928 16:01:59.137846 (+ 1275us) client_negotiation.cc:240] Negotiation successful
0928 16:01:59.138113 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1417,"thread_start_us":1281,"threads_started":1}
I20260928 16:01:59.144085 10136 tablet_service.cc:1467] Processing CreateTablet for tablet 0ed5638368e64541836d6c25b7759fdb (DEFAULT_TABLE table=test-table [id=318412fc41ef4df5b6dc1747c45bb2fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:59.145720 10136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ed5638368e64541836d6c25b7759fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:59.150569 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.124915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:37819 (local address 127.0.0.1:56612)
0928 16:01:59.138967 (+ 14052us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:59.138991 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:59.139106 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:59.143209 (+ 4103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:59.143213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:59.143238 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:59.143523 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.143535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.146515 (+ 2980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.146520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.147982 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.147995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.149762 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.149768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.149960 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.149995 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 16:01:59.150099 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 16:01:59.150241 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13917,"thread_start_us":21283,"threads_started":1}
I20260928 16:01:59.150637 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.125164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.22.125.194:37819)
0928 16:01:59.142467 (+ 17303us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.142479 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.142516 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:01:59.142639 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.142651 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.142962 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.143203 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:59.143704 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.145835 (+ 2131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.148206 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.149577 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.149745 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.149923 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.150166 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.150330 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17029,"thread_start_us":20809,"threads_started":1}
I20260928 16:01:59.151268 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.126328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:36341 (local address 127.0.0.1:43622)
0928 16:01:59.140252 (+ 13924us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:59.140272 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:59.140387 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:59.141399 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:59.141405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:59.141432 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:59.141838 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.141851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.145653 (+ 3802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.145660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.148323 (+ 2663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.148337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.150409 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.150416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.150600 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.150645 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 16:01:59.150744 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 16:01:59.150876 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13809,"mutex_wait_us":1,"thread_start_us":13630,"threads_started":1}
I20260928 16:01:59.151568 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.126865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.22.125.195:36341)
0928 16:01:59.139752 (+ 12887us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.139764 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.140529 (+ 765us) server_negotiation.cc:407] Connection header received
0928 16:01:59.140644 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.140652 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.140915 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.141119 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:59.143779 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.145453 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.148492 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.149895 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.150094 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.150253 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.151068 (+ 815us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.151235 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12679,"thread_start_us":12515,"threads_started":1}
I20260928 16:01:59.154337 10270 tablet_service.cc:1467] Processing CreateTablet for tablet 0ed5638368e64541836d6c25b7759fdb (DEFAULT_TABLE table=test-table [id=318412fc41ef4df5b6dc1747c45bb2fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:59.155761 10270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ed5638368e64541836d6c25b7759fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:59.155682 10403 tablet_service.cc:1467] Processing CreateTablet for tablet 0ed5638368e64541836d6c25b7759fdb (DEFAULT_TABLE table=test-table [id=318412fc41ef4df5b6dc1747c45bb2fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:01:59.156777 10403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ed5638368e64541836d6c25b7759fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:01:59.161552 10486 tablet_bootstrap.cc:492] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Bootstrap starting.
I20260928 16:01:59.165802 10486 tablet_bootstrap.cc:654] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:59.167028 10486 log.cc:824] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:59.167596 10487 tablet_bootstrap.cc:492] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Bootstrap starting.
I20260928 16:01:59.170569 10486 tablet_bootstrap.cc:492] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: No bootstrap required, opened a new log
I20260928 16:01:59.170722 10489 tablet_bootstrap.cc:492] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Bootstrap starting.
I20260928 16:01:59.170917 10486 ts_tablet_manager.cc:1397] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260928 16:01:59.171458 10487 tablet_bootstrap.cc:654] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:59.172739 10487 log.cc:824] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:59.174139 10489 tablet_bootstrap.cc:654] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Neither blocks nor log segments found. Creating new log.
I20260928 16:01:59.175046 10489 log.cc:824] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Log is configured to *not* fsync() on all Append() calls
I20260928 16:01:59.175830 10487 tablet_bootstrap.cc:492] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: No bootstrap required, opened a new log
I20260928 16:01:59.176100 10487 ts_tablet_manager.cc:1397] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260928 16:01:59.177587 10489 tablet_bootstrap.cc:492] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: No bootstrap required, opened a new log
I20260928 16:01:59.177838 10489 ts_tablet_manager.cc:1397] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260928 16:01:59.181303 10486 raft_consensus.cc:348] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.181728 10486 raft_consensus.cc:374] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:59.181874 10486 raft_consensus.cc:729] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dbe0aece9c94e7289be75920be49ac4, State: Initialized, Role: FOLLOWER
I20260928 16:01:59.182475 10486 consensus_queue.cc:260] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.184140 10486 ts_tablet_manager.cc:1428] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260928 16:01:59.185755 10489 raft_consensus.cc:348] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.186220 10489 raft_consensus.cc:374] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:59.186393 10489 raft_consensus.cc:729] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8be1376c5346d59125b6fa7b15e82d, State: Initialized, Role: FOLLOWER
I20260928 16:01:59.186857 10487 raft_consensus.cc:348] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.187181 10487 raft_consensus.cc:374] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:01:59.186993 10489 consensus_queue.cc:260] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.187310 10487 raft_consensus.cc:729] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6cc44e3ef24b868a76fc18238d558d, State: Initialized, Role: FOLLOWER
I20260928 16:01:59.187923 10487 consensus_queue.cc:260] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.189538 10489 ts_tablet_manager.cc:1428] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260928 16:01:59.189935 10335 heartbeater.cc:503] Master 127.22.125.254:42761 was elected leader, sending a full tablet report...
I20260928 16:01:59.191434 10468 heartbeater.cc:503] Master 127.22.125.254:42761 was elected leader, sending a full tablet report...
I20260928 16:01:59.192026 10487 ts_tablet_manager.cc:1428] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260928 16:01:59.203091 10493 raft_consensus.cc:482] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:01:59.203373 10493 raft_consensus.cc:504] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.204955 10493 leader_election.cc:288] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dbe0aece9c94e7289be75920be49ac4 (127.22.125.193:39869), df6cc44e3ef24b868a76fc18238d558d (127.22.125.195:36341)
I20260928 16:01:59.213286 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.206095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:41663 (local address 127.22.125.195:36341)
0928 16:01:59.206313 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.206320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.206838 (+ 518us) server_negotiation.cc:407] Connection header received
0928 16:01:59.206965 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.206969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.207021 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.207090 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:59.207602 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.208571 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.210852 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.212072 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.212214 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.212287 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.212912 (+ 625us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.213041 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:01:59.213289 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.205281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:60071 (local address 127.22.125.193:39869)
0928 16:01:59.205472 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.205477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.205998 (+ 521us) server_negotiation.cc:407] Connection header received
0928 16:01:59.206339 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.206348 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.206442 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.206548 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:01:59.207203 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.208253 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.210815 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.212061 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.212214 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.212287 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.212912 (+ 625us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.213041 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:01:59.213511 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.205996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:36341 (local address 127.22.125.194:41663)
0928 16:01:59.206701 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:59.206721 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:59.206851 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:59.207129 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:59.207136 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:59.207166 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:59.207448 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.207455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.208687 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.208692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.210698 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.210712 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.212366 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.212373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.212623 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.212774 (+ 151us) client_negotiation.cc:769] Sending connection context
0928 16:01:59.212873 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 16:01:59.213057 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":120,"thread_start_us":397,"threads_started":1}
I20260928 16:01:59.213692 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.205281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:39869 (local address 127.22.125.194:60071)
0928 16:01:59.205874 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0928 16:01:59.205890 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 16:01:59.206087 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:01:59.206584 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:01:59.206590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:01:59.206620 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:01:59.207043 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.207052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.208376 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.208380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.210627 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:01:59.210643 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.212367 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.212373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:01:59.212623 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.212773 (+ 150us) client_negotiation.cc:769] Sending connection context
0928 16:01:59.212873 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 16:01:59.213057 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":122,"thread_start_us":327,"threads_started":1}
I20260928 16:01:59.214715 10156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed5638368e64541836d6c25b7759fdb" candidate_uuid: "db8be1376c5346d59125b6fa7b15e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbe0aece9c94e7289be75920be49ac4" is_pre_election: true
I20260928 16:01:59.214716 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed5638368e64541836d6c25b7759fdb" candidate_uuid: "db8be1376c5346d59125b6fa7b15e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6cc44e3ef24b868a76fc18238d558d" is_pre_election: true
I20260928 16:01:59.215286 10423 raft_consensus.cc:2454] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8be1376c5346d59125b6fa7b15e82d in term 0.
I20260928 16:01:59.215286 10156 raft_consensus.cc:2454] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8be1376c5346d59125b6fa7b15e82d in term 0.
I20260928 16:01:59.215946 10223 leader_election.cc:302] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dbe0aece9c94e7289be75920be49ac4, db8be1376c5346d59125b6fa7b15e82d; no voters:
I20260928 16:01:59.216341 10493 raft_consensus.cc:2781] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:01:59.216486 10493 raft_consensus.cc:482] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:01:59.216610 10493 raft_consensus.cc:3037] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:59.218756 10493 raft_consensus.cc:504] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.219300 10493 leader_election.cc:288] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [CANDIDATE]: Term 1 election: Requested vote from peers 5dbe0aece9c94e7289be75920be49ac4 (127.22.125.193:39869), df6cc44e3ef24b868a76fc18238d558d (127.22.125.195:36341)
I20260928 16:01:59.219801 10156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed5638368e64541836d6c25b7759fdb" candidate_uuid: "db8be1376c5346d59125b6fa7b15e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbe0aece9c94e7289be75920be49ac4"
I20260928 16:01:59.219864 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed5638368e64541836d6c25b7759fdb" candidate_uuid: "db8be1376c5346d59125b6fa7b15e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6cc44e3ef24b868a76fc18238d558d"
I20260928 16:01:59.220057 10423 raft_consensus.cc:3037] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:59.220018 10156 raft_consensus.cc:3037] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:01:59.223238 10423 raft_consensus.cc:2454] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8be1376c5346d59125b6fa7b15e82d in term 1.
I20260928 16:01:59.223238 10156 raft_consensus.cc:2454] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8be1376c5346d59125b6fa7b15e82d in term 1.
I20260928 16:01:59.223605 10223 leader_election.cc:302] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dbe0aece9c94e7289be75920be49ac4, db8be1376c5346d59125b6fa7b15e82d; no voters:
I20260928 16:01:59.223888 10493 raft_consensus.cc:2781] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:01:59.224345 10493 raft_consensus.cc:686] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 1 LEADER]: Becoming Leader. State: Replica: db8be1376c5346d59125b6fa7b15e82d, State: Running, Role: LEADER
I20260928 16:01:59.224869 10493 consensus_queue.cc:237] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } }
I20260928 16:01:59.229131 10012 catalog_manager.cc:5495] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d reported cstate change: term changed from 0 to 1, leader changed from <none> to db8be1376c5346d59125b6fa7b15e82d (127.22.125.194). New cstate: current_term: 1 leader_uuid: "db8be1376c5346d59125b6fa7b15e82d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db8be1376c5346d59125b6fa7b15e82d" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 37819 } health_report { overall_health: HEALTHY } } }
I20260928 16:01:59.256290 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.251241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.22.125.194:37819)
0928 16:01:59.251435 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.251439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.251534 (+ 95us) server_negotiation.cc:407] Connection header received
0928 16:01:59.251714 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.251717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.251806 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.251941 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:01:59.252334 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.253057 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.254392 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.254772 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.254877 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.255304 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:01:59.255762 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:01:59.255880 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.255966 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.256026 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 16:01:59.264233 10423 raft_consensus.cc:1261] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 1 FOLLOWER]: Refusing update from remote peer db8be1376c5346d59125b6fa7b15e82d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:59.264233 10156 raft_consensus.cc:1261] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 1 FOLLOWER]: Refusing update from remote peer db8be1376c5346d59125b6fa7b15e82d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:01:59.265182 10497 consensus_queue.cc:1035] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dbe0aece9c94e7289be75920be49ac4" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 39869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:59.265516 10493 consensus_queue.cc:1035] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [LEADER]: Connected to new peer: Peer: permanent_uuid: "df6cc44e3ef24b868a76fc18238d558d" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:01:59.276916 10502 mvcc.cc:205] Tried to move back new op lower bound from 7334343963696099328 to 7334343963549020160. Current Snapshot: MvccSnapshot[applied={T|T < 7334343963696099328}]
I20260928 16:01:59.279520 10501 mvcc.cc:205] Tried to move back new op lower bound from 7334343963696099328 to 7334343963549020160. Current Snapshot: MvccSnapshot[applied={T|T < 7334343963696099328}]
I20260928 16:01:59.295002 10012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:59.295289 10012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:01:59.298817 10012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6c55bdf5e154a828392495eeefad442: Sending DeleteTablet for 3 replicas of tablet 0ed5638368e64541836d6c25b7759fdb
I20260928 16:01:59.299932 10136 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ed5638368e64541836d6c25b7759fdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:59 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:34732
I20260928 16:01:59.300064 10403 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ed5638368e64541836d6c25b7759fdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:59 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:43622
I20260928 16:01:59.300256 10270 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ed5638368e64541836d6c25b7759fdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:01:59 UTC) from {username='kudu', principal='kudu/127.22.125.254@KRBTEST.COM'} at 127.0.0.1:56612
I20260928 16:01:59.305594 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 10073
I20260928 16:01:59.313737 10512 tablet_replica.cc:330] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: stopping tablet replica
I20260928 16:01:59.314358 10512 raft_consensus.cc:2229] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:01:59.315241 10512 raft_consensus.cc:2258] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:59.316098 10511 tablet_replica.cc:330] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: stopping tablet replica
I20260928 16:01:59.316540 10511 raft_consensus.cc:2229] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:59.317071 10511 raft_consensus.cc:2258] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:59.319329 10512 ts_tablet_manager.cc:1905] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:59.321724 10511 ts_tablet_manager.cc:1905] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:01:59.328101 10512 ts_tablet_manager.cc:1918] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:59.328377 10512 log.cc:1196] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/0ed5638368e64541836d6c25b7759fdb
I20260928 16:01:59.328770 10511 ts_tablet_manager.cc:1918] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:59.328967 10511 log.cc:1196] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/0ed5638368e64541836d6c25b7759fdb
I20260928 16:01:59.329105 10512 ts_tablet_manager.cc:1939] T 0ed5638368e64541836d6c25b7759fdb P db8be1376c5346d59125b6fa7b15e82d: Deleting consensus metadata
I20260928 16:01:59.329535 10511 ts_tablet_manager.cc:1939] T 0ed5638368e64541836d6c25b7759fdb P df6cc44e3ef24b868a76fc18238d558d: Deleting consensus metadata
I20260928 16:01:59.331013 10510 tablet_replica.cc:330] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: stopping tablet replica
I20260928 16:01:59.331228 9997 catalog_manager.cc:4841] TS db8be1376c5346d59125b6fa7b15e82d (127.22.125.194:37819): tablet 0ed5638368e64541836d6c25b7759fdb (table test-table [id=318412fc41ef4df5b6dc1747c45bb2fc]) successfully deleted
I20260928 16:01:59.331785 9996 catalog_manager.cc:4841] TS df6cc44e3ef24b868a76fc18238d558d (127.22.125.195:36341): tablet 0ed5638368e64541836d6c25b7759fdb (table test-table [id=318412fc41ef4df5b6dc1747c45bb2fc]) successfully deleted
I20260928 16:01:59.331825 10510 raft_consensus.cc:2229] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:01:59.332367 10510 raft_consensus.cc:2258] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:01:59.334870 10510 ts_tablet_manager.cc:1905] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 16:01:59 dist-test-slave-9cqd krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.193@KRBTEST.COM
I20260928 16:01:59.341149 10510 ts_tablet_manager.cc:1918] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:01:59.341318 10510 log.cc:1196] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/0ed5638368e64541836d6c25b7759fdb
I20260928 16:01:59.341873 10510 ts_tablet_manager.cc:1939] T 0ed5638368e64541836d6c25b7759fdb P 5dbe0aece9c94e7289be75920be49ac4: Deleting consensus metadata
I20260928 16:01:59.343505 9996 catalog_manager.cc:4841] TS 5dbe0aece9c94e7289be75920be49ac4 (127.22.125.193:39869): tablet 0ed5638368e64541836d6c25b7759fdb (table test-table [id=318412fc41ef4df5b6dc1747c45bb2fc]) successfully deleted
I20260928 16:01:59.345079 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.307020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.22.125.193:39869)
0928 16:01:59.307241 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.307251 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.324177 (+ 16926us) server_negotiation.cc:407] Connection header received
0928 16:01:59.324689 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.324694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.324790 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.324896 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:59.326718 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.327893 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.331331 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.332072 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.332231 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.337651 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:59.337663 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:59.337680 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:59.337726 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:59.341869 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.342792 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.342806 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.342810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.342964 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.343458 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.343462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.343465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.343891 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:59.344183 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.344600 (+ 417us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.344760 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:01:59.345841 10196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:59.434844 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 10073
I20260928 16:01:59.458945 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 10207
Sep 28 16:01:59 dist-test-slave-9cqd krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.194@KRBTEST.COM
I20260928 16:01:59.476646 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.459859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.22.125.194:37819)
0928 16:01:59.460028 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.460032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.460095 (+ 63us) server_negotiation.cc:407] Connection header received
0928 16:01:59.460182 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.460186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.460252 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.460333 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:59.462028 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.462812 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.464723 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.465149 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.465242 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.470601 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:59.470616 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:59.470634 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:59.470670 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:59.473714 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.474511 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.474518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.474520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.474750 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.475187 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.475190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.475192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.475544 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:59.475780 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.476151 (+ 371us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.476314 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 16:01:59.477444 10330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:59.588979 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 10207
I20260928 16:01:59.608533 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 10340
Sep 28 16:01:59 dist-test-slave-9cqd krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.195@KRBTEST.COM
I20260928 16:01:59.624028 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:01:59.609252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.22.125.195:36341)
0928 16:01:59.609452 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 16:01:59.609456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:01:59.609474 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:01:59.609621 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:01:59.609625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:01:59.609697 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:01:59.609778 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:01:59.611065 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.611766 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.613109 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:01:59.613488 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:01:59.613578 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:01:59.617727 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:01:59.617741 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:01:59.617756 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:01:59.617787 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:01:59.621672 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.622309 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.622316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.622318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.622457 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:01:59.622763 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:01:59.622766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:01:59.622768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:01:59.623115 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:01:59.623333 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0928 16:01:59.623539 (+ 206us) server_negotiation.cc:299] Negotiation successful
0928 16:01:59.623692 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:01:59.624765 10463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:59.714892 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 10340
I20260928 16:01:59.731431 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 9980
I20260928 16:01:59.732498 10042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:01:59.840785 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 9980
2026-09-28T16:01:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4199 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T16:01:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:01:59Z Disabled control of system clock
I20260928 16:01:59.915026 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45903
--webserver_interface=127.22.125.254
--webserver_port=0
--builtin_ntp_servers=127.22.125.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:45903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:02:00.240703 10521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:00.241000 10521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:00.241142 10521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:00.248364 10521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:00.248479 10521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:00.248543 10521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:00.248648 10521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:00.248709 10521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:00.257597 10521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:45903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:00.258793 10521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:00.260540 10521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:00.271991 10526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:00.272084 10527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:00.272279 10529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:00.273056 10521 server_base.cc:1034] running on GCE node
I20260928 16:02:00.273700 10521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:00.274926 10521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:00.276131 10521 hybrid_clock.cc:648] HybridClock initialized: now 1790611320276093 us; error 47 us; skew 500 ppm
I20260928 16:02:00.276538 10521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:02:00.278605 10521 webserver.cc:460] Webserver started at http://127.22.125.254:40297/ using document root <none> and password file <none>
I20260928 16:02:00.279337 10521 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:00.279457 10521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:00.279747 10521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:00.282506 10521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "39379f12f50b403bb221ac9a77135111"
format_stamp: "Formatted at 2026-09-28 16:02:00 on dist-test-slave-9cqd"
server_key: "d790c24a36d3a176c68900753f24f1ab"
server_key_iv: "781fd87e683bae4b56a1242f83704fb0"
server_key_version: "encryptionkey@0"
I20260928 16:02:00.283284 10521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "39379f12f50b403bb221ac9a77135111"
format_stamp: "Formatted at 2026-09-28 16:02:00 on dist-test-slave-9cqd"
server_key: "d790c24a36d3a176c68900753f24f1ab"
server_key_iv: "781fd87e683bae4b56a1242f83704fb0"
server_key_version: "encryptionkey@0"
I20260928 16:02:00.288141 10521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260928 16:02:00.291307 10535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:00.292625 10521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 16:02:00.292807 10521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "39379f12f50b403bb221ac9a77135111"
format_stamp: "Formatted at 2026-09-28 16:02:00 on dist-test-slave-9cqd"
server_key: "d790c24a36d3a176c68900753f24f1ab"
server_key_iv: "781fd87e683bae4b56a1242f83704fb0"
server_key_version: "encryptionkey@0"
I20260928 16:02:00.293017 10521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:00.307544 10521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:00.308344 10521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:00.308609 10521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:00.326542 10521 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:45903
I20260928 16:02:00.326561 10586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:45903 every 8 connection(s)
I20260928 16:02:00.327836 10521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:00.331373 10587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:00.335701 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 10521
I20260928 16:02:00.335896 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:00.336472 23031 external_mini_cluster.cc:1428] Setting key fdbae8601cf98b5ceca32a5f150edb81
I20260928 16:02:00.342651 10587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111: Bootstrap starting.
I20260928 16:02:00.346138 10587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:00.347337 10587 log.cc:824] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:00.350303 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:00.338272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.22.125.254:45903)
0928 16:02:00.339207 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:00.339226 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:00.339266 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:02:00.341201 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:00.341229 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:00.341643 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:00.341943 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:00.342540 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:00.343873 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:00.345779 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:00.346509 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:00.346639 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:00.347815 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:00.347822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:00.347854 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:00.348676 (+ 822us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:00.348775 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:00.348795 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:00.348884 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:00.349096 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 16:02:00.349446 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":363,"threads_started":1}
I20260928 16:02:00.350394 10587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111: No bootstrap required, opened a new log
I20260928 16:02:00.357506 10587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39379f12f50b403bb221ac9a77135111" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45903 } }
I20260928 16:02:00.357872 10587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:00.357991 10587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39379f12f50b403bb221ac9a77135111, State: Initialized, Role: FOLLOWER
I20260928 16:02:00.358511 10587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39379f12f50b403bb221ac9a77135111" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45903 } }
I20260928 16:02:00.358759 10587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:00.358909 10587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:00.359077 10587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:00.361251 10587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39379f12f50b403bb221ac9a77135111" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45903 } }
I20260928 16:02:00.361732 10587 leader_election.cc:302] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39379f12f50b403bb221ac9a77135111; no voters:
I20260928 16:02:00.362278 10587 leader_election.cc:288] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:00.362443 10592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:00.363152 10592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [term 1 LEADER]: Becoming Leader. State: Replica: 39379f12f50b403bb221ac9a77135111, State: Running, Role: LEADER
I20260928 16:02:00.363737 10592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39379f12f50b403bb221ac9a77135111" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45903 } }
I20260928 16:02:00.364391 10587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:00.366873 10593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39379f12f50b403bb221ac9a77135111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39379f12f50b403bb221ac9a77135111" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45903 } } }
I20260928 16:02:00.367172 10594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39379f12f50b403bb221ac9a77135111. Latest consensus state: current_term: 1 leader_uuid: "39379f12f50b403bb221ac9a77135111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39379f12f50b403bb221ac9a77135111" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45903 } } }
I20260928 16:02:00.367400 10593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:00.367486 10594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:00.369068 10601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:00.373471 10601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:00.381403 10601 catalog_manager.cc:1347] Generated new cluster ID: 23d10eefdf6d4e9380b2d89431b158d1
I20260928 16:02:00.381512 10601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:00.417436 10601 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:00.418746 10601 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:00.426963 10601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111: Generated new TSK 0
I20260928 16:02:00.427747 10601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:00.437315 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.193:0
--local_ip_for_outbound_sockets=127.22.125.193
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45903
--builtin_ntp_servers=127.22.125.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 16:02:00.767203 10611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:00.767455 10611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:00.767618 10611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:00.774664 10611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:00.774746 10611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:00.774878 10611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.193
I20260928 16:02:00.783711 10611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.125.193
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.193
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:00.784967 10611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:00.786692 10611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:00.799381 10616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:00.799391 10617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:00.799935 10619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:00.800331 10611 server_base.cc:1034] running on GCE node
I20260928 16:02:00.800941 10611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:00.801793 10611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:00.803032 10611 hybrid_clock.cc:648] HybridClock initialized: now 1790611320802990 us; error 50 us; skew 500 ppm
I20260928 16:02:00.803337 10611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:02:00.805496 10611 webserver.cc:460] Webserver started at http://127.22.125.193:38399/ using document root <none> and password file <none>
I20260928 16:02:00.806174 10611 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:00.806299 10611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:00.806602 10611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:00.809389 10611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data/instance:
uuid: "b8637a4b055e408494a4dc0abe44f117"
format_stamp: "Formatted at 2026-09-28 16:02:00 on dist-test-slave-9cqd"
server_key: "682be26588d685077cb334db1c7f5423"
server_key_iv: "d217a1951ce684aba0b1de3849b46c0f"
server_key_version: "encryptionkey@0"
I20260928 16:02:00.810154 10611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance:
uuid: "b8637a4b055e408494a4dc0abe44f117"
format_stamp: "Formatted at 2026-09-28 16:02:00 on dist-test-slave-9cqd"
server_key: "682be26588d685077cb334db1c7f5423"
server_key_iv: "d217a1951ce684aba0b1de3849b46c0f"
server_key_version: "encryptionkey@0"
I20260928 16:02:00.814993 10611 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260928 16:02:00.818334 10625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:00.819615 10611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 16:02:00.819802 10611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal
uuid: "b8637a4b055e408494a4dc0abe44f117"
format_stamp: "Formatted at 2026-09-28 16:02:00 on dist-test-slave-9cqd"
server_key: "682be26588d685077cb334db1c7f5423"
server_key_iv: "d217a1951ce684aba0b1de3849b46c0f"
server_key_version: "encryptionkey@0"
I20260928 16:02:00.820012 10611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:00.834856 10611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:00.835600 10611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:00.835909 10611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:00.836858 10611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:02:00.838233 10611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:02:00.838352 10611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:00.838474 10611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:02:00.838557 10611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:00.871287 10611 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.193:42465
I20260928 16:02:00.871342 10737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.193:42465 every 8 connection(s)
I20260928 16:02:00.872737 10611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/data/info.pb
I20260928 16:02:00.873811 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 10611
I20260928 16:02:00.874027 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal/instance
I20260928 16:02:00.874572 23031 external_mini_cluster.cc:1428] Setting key 4201c84fa2fcaf2d56991ef136557e09
I20260928 16:02:00.877696 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.194:0
--local_ip_for_outbound_sockets=127.22.125.194
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45903
--builtin_ntp_servers=127.22.125.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:02:00.887137 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:00.875563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.193:56573 (local address 127.22.125.254:45903)
0928 16:02:00.876032 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:00.876055 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:00.877392 (+ 1337us) server_negotiation.cc:407] Connection header received
0928 16:02:00.879283 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:00.879288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:00.879372 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:00.879472 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:00.880678 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:00.881403 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:00.883659 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:00.884438 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:00.884571 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:00.886195 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:00.886204 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:00.886212 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:00.886236 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:00.886286 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:00.886304 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:00.886391 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:00.886587 (+ 196us) server_negotiation.cc:299] Negotiation successful
0928 16:02:00.886693 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260928 16:02:00.887952 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:00.876357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45903 (local address 127.22.125.193:56573)
0928 16:02:00.877250 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:00.877287 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:00.879023 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:00.879702 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:00.879710 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:00.879793 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:00.880519 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:00.880537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:00.881592 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:00.881596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:00.883497 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:00.883513 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:00.884713 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:00.884722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:00.885078 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:00.885869 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:02:00.885873 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:00.885912 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:02:00.885995 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:00.886404 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:00.886412 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:00.886417 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 16:02:00.886549 (+ 132us) client_negotiation.cc:240] Negotiation successful
0928 16:02:00.886841 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":292,"threads_started":1}
I20260928 16:02:00.889429 10738 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45903
I20260928 16:02:00.889714 10738 heartbeater.cc:464] Registering TS with master...
I20260928 16:02:00.890431 10738 heartbeater.cc:511] Master 127.22.125.254:45903 requested a full tablet report, sending...
I20260928 16:02:00.892470 10552 ts_manager.cc:194] Registered new tserver with Master: b8637a4b055e408494a4dc0abe44f117 (127.22.125.193:42465)
I20260928 16:02:00.893879 10552 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.193:56573
W20260928 16:02:01.195022 10742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:01.195286 10742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:01.195432 10742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:01.202623 10742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:01.202711 10742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:01.202910 10742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.194
I20260928 16:02:01.211764 10742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.125.194
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.194
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:01.213002 10742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:01.214721 10742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:01.227414 10748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:01.227466 10747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:01.227934 10750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:01.228497 10742 server_base.cc:1034] running on GCE node
I20260928 16:02:01.229065 10742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:01.229933 10742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:01.231154 10742 hybrid_clock.cc:648] HybridClock initialized: now 1790611321231108 us; error 54 us; skew 500 ppm
I20260928 16:02:01.231458 10742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:02:01.233608 10742 webserver.cc:460] Webserver started at http://127.22.125.194:46075/ using document root <none> and password file <none>
I20260928 16:02:01.234269 10742 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:01.234395 10742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:01.234695 10742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:01.237491 10742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data/instance:
uuid: "8406068d10e14ee998e3c75a7ed1ebb7"
format_stamp: "Formatted at 2026-09-28 16:02:01 on dist-test-slave-9cqd"
server_key: "22376450b8f018619aeccd1b7888c7d3"
server_key_iv: "64ae858db3b5fbf72b2e4a61bd0483ff"
server_key_version: "encryptionkey@0"
I20260928 16:02:01.238263 10742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance:
uuid: "8406068d10e14ee998e3c75a7ed1ebb7"
format_stamp: "Formatted at 2026-09-28 16:02:01 on dist-test-slave-9cqd"
server_key: "22376450b8f018619aeccd1b7888c7d3"
server_key_iv: "64ae858db3b5fbf72b2e4a61bd0483ff"
server_key_version: "encryptionkey@0"
I20260928 16:02:01.243031 10742 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 16:02:01.246304 10756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:01.247452 10742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 16:02:01.247613 10742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal
uuid: "8406068d10e14ee998e3c75a7ed1ebb7"
format_stamp: "Formatted at 2026-09-28 16:02:01 on dist-test-slave-9cqd"
server_key: "22376450b8f018619aeccd1b7888c7d3"
server_key_iv: "64ae858db3b5fbf72b2e4a61bd0483ff"
server_key_version: "encryptionkey@0"
I20260928 16:02:01.247823 10742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:01.275435 10742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:01.276263 10742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:01.276548 10742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:01.277482 10742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:02:01.278941 10742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:02:01.279062 10742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:01.279192 10742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:02:01.279277 10742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:01.311308 10742 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.194:42267
I20260928 16:02:01.311367 10868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.194:42267 every 8 connection(s)
I20260928 16:02:01.312770 10742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/data/info.pb
I20260928 16:02:01.321950 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 10742
I20260928 16:02:01.322130 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal/instance
I20260928 16:02:01.322713 23031 external_mini_cluster.cc:1428] Setting key 081d4e7a92da324bb0c6e73152a2edf9
I20260928 16:02:01.325637 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.315209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:54047 (local address 127.22.125.254:45903)
0928 16:02:01.315387 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.315391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.316745 (+ 1354us) server_negotiation.cc:407] Connection header received
0928 16:02:01.318651 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.318656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.318744 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.318863 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:01.319891 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.320679 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.322582 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.323021 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.323161 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.324836 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:01.324843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:01.324847 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:01.324868 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:01.324906 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:01.324924 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:01.325020 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:01.325224 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 16:02:01.325315 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 16:02:01.325902 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.125.195:0
--local_ip_for_outbound_sockets=127.22.125.195
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45903
--builtin_ntp_servers=127.22.125.212:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 16:02:01.326711 10871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.315688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45903 (local address 127.22.125.194:54047)
0928 16:02:01.316602 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:01.316638 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:01.318391 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:01.319099 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:01.319107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:01.319188 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:01.319723 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.319738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.320847 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.320851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.322438 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.322455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.323308 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.323318 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.323604 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.324446 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:02:01.324466 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:01.324522 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:02:01.324619 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:01.325022 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:01.325031 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:01.325036 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 16:02:01.325174 (+ 138us) client_negotiation.cc:240] Negotiation successful
0928 16:02:01.325453 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":305,"threads_started":1}
I20260928 16:02:01.328552 10869 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45903
I20260928 16:02:01.328845 10869 heartbeater.cc:464] Registering TS with master...
I20260928 16:02:01.329659 10869 heartbeater.cc:511] Master 127.22.125.254:45903 requested a full tablet report, sending...
I20260928 16:02:01.331205 10552 ts_manager.cc:194] Registered new tserver with Master: 8406068d10e14ee998e3c75a7ed1ebb7 (127.22.125.194:42267)
I20260928 16:02:01.332446 10552 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.194:54047
W20260928 16:02:01.646713 10872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:01.646963 10872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:01.647121 10872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:01.654441 10872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:01.654533 10872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:01.654731 10872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.125.195
I20260928 16:02:01.663590 10872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.125.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.125.195
--webserver_port=0
--tserver_master_addrs=127.22.125.254:45903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.125.195
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:01.664922 10872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:01.666714 10872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:01.679914 10877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:01.679989 10878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:01.679962 10880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:01.680738 10872 server_base.cc:1034] running on GCE node
I20260928 16:02:01.681296 10872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:01.682245 10872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:01.683454 10872 hybrid_clock.cc:648] HybridClock initialized: now 1790611321683400 us; error 42 us; skew 500 ppm
I20260928 16:02:01.683804 10872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 16:02:01.686254 10872 webserver.cc:460] Webserver started at http://127.22.125.195:38495/ using document root <none> and password file <none>
I20260928 16:02:01.686928 10872 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:01.687044 10872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:01.687368 10872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:01.690274 10872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data/instance:
uuid: "4f2484f4e25a4351ae70be15e845fb17"
format_stamp: "Formatted at 2026-09-28 16:02:01 on dist-test-slave-9cqd"
server_key: "4f5b4b55de0a968df1db7b436836cc3e"
server_key_iv: "38d1ff17fd63fb7f92db221866d9cb91"
server_key_version: "encryptionkey@0"
I20260928 16:02:01.691035 10872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance:
uuid: "4f2484f4e25a4351ae70be15e845fb17"
format_stamp: "Formatted at 2026-09-28 16:02:01 on dist-test-slave-9cqd"
server_key: "4f5b4b55de0a968df1db7b436836cc3e"
server_key_iv: "38d1ff17fd63fb7f92db221866d9cb91"
server_key_version: "encryptionkey@0"
I20260928 16:02:01.695915 10872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260928 16:02:01.699173 10886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:01.700538 10872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:01.700707 10872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal
uuid: "4f2484f4e25a4351ae70be15e845fb17"
format_stamp: "Formatted at 2026-09-28 16:02:01 on dist-test-slave-9cqd"
server_key: "4f5b4b55de0a968df1db7b436836cc3e"
server_key_iv: "38d1ff17fd63fb7f92db221866d9cb91"
server_key_version: "encryptionkey@0"
I20260928 16:02:01.700903 10872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:01.731055 10872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:01.731909 10872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:01.732237 10872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:01.733206 10872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 16:02:01.734608 10872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 16:02:01.734699 10872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:01.734843 10872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 16:02:01.734930 10872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:01.766939 10872 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.195:38493
I20260928 16:02:01.767022 10998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.195:38493 every 8 connection(s)
I20260928 16:02:01.768241 10872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/data/info.pb
I20260928 16:02:01.770829 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 10872
I20260928 16:02:01.771018 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal/instance
I20260928 16:02:01.771433 23031 external_mini_cluster.cc:1428] Setting key 6571617ff420bca7dbf15169421ce614
I20260928 16:02:01.780269 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.770814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.195:49571 (local address 127.22.125.254:45903)
0928 16:02:01.771044 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.771052 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.772674 (+ 1622us) server_negotiation.cc:407] Connection header received
0928 16:02:01.774349 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.774353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.774418 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.774497 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:01.775462 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.776270 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.777720 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.778130 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.778219 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.779498 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:01.779503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:01.779506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:01.779521 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:01.779551 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:01.779562 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:01.779632 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:01.779874 (+ 242us) server_negotiation.cc:299] Negotiation successful
0928 16:02:01.779947 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 16:02:01.781219 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.771430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:45903 (local address 127.22.125.195:49571)
0928 16:02:01.772530 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:01.772574 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:01.774124 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:01.774691 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:01.774699 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:01.774783 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:01.775331 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.775347 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.776388 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.776391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.777599 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.777615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.778304 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.778309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.778506 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.779170 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 16:02:01.779172 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:01.779209 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 16:02:01.779309 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:01.779675 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:01.779684 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:01.779691 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 16:02:01.779847 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 16:02:01.780228 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":341,"threads_started":1}
I20260928 16:02:01.782521 10999 heartbeater.cc:347] Connected to a master server at 127.22.125.254:45903
I20260928 16:02:01.782791 10999 heartbeater.cc:464] Registering TS with master...
I20260928 16:02:01.783514 10999 heartbeater.cc:511] Master 127.22.125.254:45903 requested a full tablet report, sending...
I20260928 16:02:01.785041 10552 ts_manager.cc:194] Registered new tserver with Master: 4f2484f4e25a4351ae70be15e845fb17 (127.22.125.195:38493)
I20260928 16:02:01.786105 10552 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.125.195:49571
I20260928 16:02:01.786327 23031 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 16:02:01.796677 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.790909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.22.125.254:45903)
0928 16:02:01.791076 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.791082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.791554 (+ 472us) server_negotiation.cc:407] Connection header received
0928 16:02:01.791735 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.791741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.791814 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.791913 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:01.792408 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.793079 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.794375 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.794785 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.794914 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.795966 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:01.795971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:01.795974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:01.795990 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:01.796020 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:01.796031 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:01.796147 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:01.796284 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 16:02:01.796368 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 16:02:01.800017 10552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 16:02:01.806803 10552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 16:02:01.853037 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.818653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.22.125.194:42267)
0928 16:02:01.843679 (+ 25026us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.843686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.844397 (+ 711us) server_negotiation.cc:407] Connection header received
0928 16:02:01.844808 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.844816 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.845036 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.845203 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:02:01.846166 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.847738 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.850384 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.851874 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.852177 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.852388 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:01.852524 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 16:02:01.852703 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24881,"thread_start_us":25025,"threads_started":1}
I20260928 16:02:01.853130 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.817778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:42465 (local address 127.0.0.1:46296)
0928 16:02:01.842458 (+ 24680us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:01.842522 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:01.842756 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:01.844353 (+ 1597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:01.844361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:01.844394 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:02:01.844825 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.844839 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.847406 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.847412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.849703 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.849714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.851769 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.851775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.852002 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.852593 (+ 591us) client_negotiation.cc:769] Sending connection context
0928 16:02:01.852675 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 16:02:01.852794 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24308,"mutex_wait_us":4,"thread_start_us":26250,"threads_started":1}
I20260928 16:02:01.853660 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.818035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.194:42267 (local address 127.0.0.1:50876)
0928 16:02:01.844272 (+ 26237us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:01.844292 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:01.844579 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:01.845519 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:01.845523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:01.845552 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:02:01.846012 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.846021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.847890 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.847896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.850185 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.850205 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.852117 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.852125 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.852289 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.852328 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 16:02:01.852473 (+ 145us) client_negotiation.cc:240] Negotiation successful
0928 16:02:01.853281 (+ 808us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26102,"mutex_wait_us":49,"thread_start_us":27269,"threads_started":1}
I20260928 16:02:01.853873 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.818027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.22.125.193:42465)
0928 16:02:01.843368 (+ 25341us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.843379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.843416 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:02:01.843528 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.843536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.843810 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.844024 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:02:01.845048 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.847189 (+ 2141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.849926 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.851247 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.851413 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.851646 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:01.853307 (+ 1661us) server_negotiation.cc:299] Negotiation successful
0928 16:02:01.853464 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25185,"thread_start_us":25090,"threads_started":1}
I20260928 16:02:01.854295 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.818356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:38493 (local address 127.0.0.1:59652)
0928 16:02:01.845559 (+ 27203us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:01.845574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:01.845778 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:01.846798 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:01.846803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:01.846835 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:02:01.847173 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.847183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.849045 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.849347 (+ 302us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.851187 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.851199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.853537 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.853546 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:01.853751 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.853782 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 16:02:01.853874 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 16:02:01.854018 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27082,"thread_start_us":26620,"threads_started":1}
I20260928 16:02:01.855063 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.819027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.22.125.195:38493)
0928 16:02:01.841907 (+ 22880us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.841918 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.845679 (+ 3761us) server_negotiation.cc:407] Connection header received
0928 16:02:01.845989 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.845997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.846224 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.846537 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:02:01.847409 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.848863 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.851352 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.852692 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.852879 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:01.853084 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:01.854569 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0928 16:02:01.854735 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22668,"thread_start_us":22499,"threads_started":1}
I20260928 16:02:01.857758 10804 tablet_service.cc:1467] Processing CreateTablet for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b (DEFAULT_TABLE table=test-table [id=fe2c1d8c11e241948ca90956331fbe56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:02:01.857756 10673 tablet_service.cc:1467] Processing CreateTablet for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b (DEFAULT_TABLE table=test-table [id=fe2c1d8c11e241948ca90956331fbe56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:02:01.859212 10804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:01.859210 10673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:01.862468 10934 tablet_service.cc:1467] Processing CreateTablet for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b (DEFAULT_TABLE table=test-table [id=fe2c1d8c11e241948ca90956331fbe56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 16:02:01.863603 10934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:01.883572 11018 tablet_bootstrap.cc:492] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Bootstrap starting.
I20260928 16:02:01.884070 11017 tablet_bootstrap.cc:492] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Bootstrap starting.
I20260928 16:02:01.887239 11017 tablet_bootstrap.cc:654] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:01.888777 11017 log.cc:824] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:01.888586 11019 tablet_bootstrap.cc:492] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Bootstrap starting.
I20260928 16:02:01.890457 11018 tablet_bootstrap.cc:654] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:01.891662 11018 log.cc:824] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:01.891947 11017 tablet_bootstrap.cc:492] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: No bootstrap required, opened a new log
I20260928 16:02:01.892261 11017 ts_tablet_manager.cc:1397] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260928 16:02:01.892531 11019 tablet_bootstrap.cc:654] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:01.893680 11019 log.cc:824] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:01.895164 11018 tablet_bootstrap.cc:492] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: No bootstrap required, opened a new log
I20260928 16:02:01.895398 11018 ts_tablet_manager.cc:1397] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.002s
I20260928 16:02:01.896162 11019 tablet_bootstrap.cc:492] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: No bootstrap required, opened a new log
I20260928 16:02:01.896538 11019 ts_tablet_manager.cc:1397] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260928 16:02:01.896682 10738 heartbeater.cc:503] Master 127.22.125.254:45903 was elected leader, sending a full tablet report...
I20260928 16:02:01.902947 11017 raft_consensus.cc:348] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:01.903486 11017 raft_consensus.cc:374] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:01.903621 11017 raft_consensus.cc:729] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8637a4b055e408494a4dc0abe44f117, State: Initialized, Role: FOLLOWER
I20260928 16:02:01.904387 11017 consensus_queue.cc:260] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:01.905678 11019 raft_consensus.cc:348] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:01.905742 11018 raft_consensus.cc:348] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:01.906149 11019 raft_consensus.cc:374] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:01.906219 11017 ts_tablet_manager.cc:1428] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260928 16:02:01.906257 11018 raft_consensus.cc:374] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:01.906298 11019 raft_consensus.cc:729] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f2484f4e25a4351ae70be15e845fb17, State: Initialized, Role: FOLLOWER
I20260928 16:02:01.906401 11018 raft_consensus.cc:729] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8406068d10e14ee998e3c75a7ed1ebb7, State: Initialized, Role: FOLLOWER
I20260928 16:02:01.906930 11019 consensus_queue.cc:260] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:01.906993 11018 consensus_queue.cc:260] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:01.908170 10869 heartbeater.cc:503] Master 127.22.125.254:45903 was elected leader, sending a full tablet report...
I20260928 16:02:01.908738 11018 ts_tablet_manager.cc:1428] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
I20260928 16:02:01.912774 10999 heartbeater.cc:503] Master 127.22.125.254:45903 was elected leader, sending a full tablet report...
I20260928 16:02:01.913201 11019 ts_tablet_manager.cc:1428] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260928 16:02:01.992719 11024 raft_consensus.cc:482] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 16:02:01.993007 11024 raft_consensus.cc:504] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:01.995004 11024 leader_election.cc:288] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8637a4b055e408494a4dc0abe44f117 (127.22.125.193:42465), 4f2484f4e25a4351ae70be15e845fb17 (127.22.125.195:38493)
I20260928 16:02:02.004799 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.995672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:41117 (local address 127.22.125.193:42465)
0928 16:02:01.995905 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.995911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.996688 (+ 777us) server_negotiation.cc:407] Connection header received
0928 16:02:01.996887 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.996893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.996987 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.997080 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:02:01.997983 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.999195 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.002395 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.003690 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.003850 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.003918 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:02.004387 (+ 469us) server_negotiation.cc:299] Negotiation successful
0928 16:02:02.004521 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:02:02.004974 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.995302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.193:42465 (local address 127.22.125.194:41117)
0928 16:02:01.996556 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:01.996578 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:01.996732 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:01.997154 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:01.997157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:01.997173 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:02:01.997827 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.997835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.999356 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.999362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:02.002156 (+ 2794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:02.002170 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.003931 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.003937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:02.004165 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.004255 (+ 90us) client_negotiation.cc:769] Sending connection context
0928 16:02:02.004356 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 16:02:02.004515 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":285,"thread_start_us":923,"threads_started":1}
I20260928 16:02:02.005434 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.995430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.125.194:34829 (local address 127.22.125.195:38493)
0928 16:02:01.995654 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:01.995660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:01.996709 (+ 1049us) server_negotiation.cc:407] Connection header received
0928 16:02:01.996913 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:01.996919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:01.996989 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:01.997075 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 16:02:01.997983 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.999227 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.002394 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.003690 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.003853 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.003919 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:02.005078 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0928 16:02:02.005203 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:02:02.005741 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:01.995376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.195:38493 (local address 127.22.125.194:34829)
0928 16:02:01.996566 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:01.996585 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:01.996766 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:01.997123 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:01.997129 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:01.997158 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 16:02:01.997820 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:01.997834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:01.999356 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:01.999362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:02.002158 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:02.002170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.003920 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.003926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:02.004167 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.004254 (+ 87us) client_negotiation.cc:769] Sending connection context
0928 16:02:02.004415 (+ 161us) client_negotiation.cc:240] Negotiation successful
0928 16:02:02.004515 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"mutex_wait_us":337,"spinlock_wait_cycles":22272,"thread_start_us":785,"threads_started":1}
I20260928 16:02:02.006065 10693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95bf8a36ecc14009a47ffc8d8f7eb13b" candidate_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8637a4b055e408494a4dc0abe44f117" is_pre_election: true
I20260928 16:02:02.006645 10693 raft_consensus.cc:2454] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8406068d10e14ee998e3c75a7ed1ebb7 in term 0.
I20260928 16:02:02.006913 10954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95bf8a36ecc14009a47ffc8d8f7eb13b" candidate_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f2484f4e25a4351ae70be15e845fb17" is_pre_election: true
I20260928 16:02:02.007356 10760 leader_election.cc:302] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8406068d10e14ee998e3c75a7ed1ebb7, b8637a4b055e408494a4dc0abe44f117; no voters:
I20260928 16:02:02.007421 10954 raft_consensus.cc:2454] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8406068d10e14ee998e3c75a7ed1ebb7 in term 0.
I20260928 16:02:02.007740 11024 raft_consensus.cc:2781] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 16:02:02.007910 11024 raft_consensus.cc:482] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 16:02:02.008059 11024 raft_consensus.cc:3037] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:02.010298 11024 raft_consensus.cc:504] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
I20260928 16:02:02.010821 11024 leader_election.cc:288] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [CANDIDATE]: Term 1 election: Requested vote from peers b8637a4b055e408494a4dc0abe44f117 (127.22.125.193:42465), 4f2484f4e25a4351ae70be15e845fb17 (127.22.125.195:38493)
I20260928 16:02:02.011205 10693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95bf8a36ecc14009a47ffc8d8f7eb13b" candidate_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8637a4b055e408494a4dc0abe44f117"
I20260928 16:02:02.011238 10954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95bf8a36ecc14009a47ffc8d8f7eb13b" candidate_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f2484f4e25a4351ae70be15e845fb17"
I20260928 16:02:02.011413 10693 raft_consensus.cc:3037] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:02.011416 10954 raft_consensus.cc:3037] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:02.014649 10693 raft_consensus.cc:2454] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8406068d10e14ee998e3c75a7ed1ebb7 in term 1.
I20260928 16:02:02.014657 10954 raft_consensus.cc:2454] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8406068d10e14ee998e3c75a7ed1ebb7 in term 1.
I20260928 16:02:02.015053 10759 leader_election.cc:302] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f2484f4e25a4351ae70be15e845fb17, 8406068d10e14ee998e3c75a7ed1ebb7; no voters:
I20260928 16:02:02.015487 11024 raft_consensus.cc:2781] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:02.015949 11024 raft_consensus.cc:686] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 1 LEADER]: Becoming Leader. State: Replica: 8406068d10e14ee998e3c75a7ed1ebb7, State: Running, Role: LEADER
I20260928 16:02:02.016479 11024 consensus_queue.cc:237] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } }
W20260928 16:02:02.019893 11000 tablet.cc:2369] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:02:02.021170 10551 catalog_manager.cc:5495] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8406068d10e14ee998e3c75a7ed1ebb7 (127.22.125.194). New cstate: current_term: 1 leader_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8406068d10e14ee998e3c75a7ed1ebb7" member_type: VOTER last_known_addr { host: "127.22.125.194" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260928 16:02:02.054476 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:02.049083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.22.125.194:42267)
0928 16:02:02.049277 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:02.049285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:02.049336 (+ 51us) server_negotiation.cc:407] Connection header received
0928 16:02:02.049580 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:02.049584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:02.049670 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:02.049822 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:02.050182 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.050958 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.052381 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.052807 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.052943 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.053404 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:02.053906 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:02.054071 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:02.054130 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 16:02:02.054205 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:02:02.062383 10954 raft_consensus.cc:1261] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 1 FOLLOWER]: Refusing update from remote peer 8406068d10e14ee998e3c75a7ed1ebb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:02:02.062383 10693 raft_consensus.cc:1261] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 1 FOLLOWER]: Refusing update from remote peer 8406068d10e14ee998e3c75a7ed1ebb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:02:02.063495 11028 consensus_queue.cc:1035] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8637a4b055e408494a4dc0abe44f117" member_type: VOTER last_known_addr { host: "127.22.125.193" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:02:02.063843 11024 consensus_queue.cc:1035] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f2484f4e25a4351ae70be15e845fb17" member_type: VOTER last_known_addr { host: "127.22.125.195" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 16:02:02.064680 10870 tablet.cc:2369] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:02:02.075094 11031 mvcc.cc:205] Tried to move back new op lower bound from 7334343975157121024 to 7334343974983430144. Current Snapshot: MvccSnapshot[applied={T|T < 7334343975157121024}]
I20260928 16:02:02.076392 11032 mvcc.cc:205] Tried to move back new op lower bound from 7334343975157121024 to 7334343974983430144. Current Snapshot: MvccSnapshot[applied={T|T < 7334343975157121024}]
I20260928 16:02:02.079856 11033 mvcc.cc:205] Tried to move back new op lower bound from 7334343975157121024 to 7334343974983430144. Current Snapshot: MvccSnapshot[applied={T|T < 7334343975157121024}]
I20260928 16:02:02.090281 10551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:02:02.090622 10551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 16:02:02.094651 10551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39379f12f50b403bb221ac9a77135111: Sending DeleteTablet for 3 replicas of tablet 95bf8a36ecc14009a47ffc8d8f7eb13b
I20260928 16:02:02.095882 10934 tablet_service.cc:1514] Processing DeleteTablet for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:02:02 UTC) from {username='slave'} at 127.0.0.1:59652
I20260928 16:02:02.096285 10673 tablet_service.cc:1514] Processing DeleteTablet for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:02:02 UTC) from {username='slave'} at 127.0.0.1:46296
I20260928 16:02:02.096984 11040 tablet_replica.cc:330] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: stopping tablet replica
I20260928 16:02:02.097018 10933 tablet_service.cc:1514] Processing DeleteTablet for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:02:02 UTC) from {username='slave'} at 127.0.0.1:59652
I20260928 16:02:02.097201 11041 tablet_replica.cc:330] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: stopping tablet replica
I20260928 16:02:02.097455 10804 tablet_service.cc:1514] Processing DeleteTablet for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 16:02:02 UTC) from {username='slave'} at 127.0.0.1:50876
I20260928 16:02:02.097580 11041 raft_consensus.cc:2229] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:02:02.097635 11040 raft_consensus.cc:2229] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 16:02:02.098065 11041 raft_consensus.cc:2258] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:02:02.098278 11040 raft_consensus.cc:2258] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:02:02.099083 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 10611
I20260928 16:02:02.100073 11041 ts_tablet_manager.cc:1905] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:02:02.100677 11040 ts_tablet_manager.cc:1905] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:02:02.107702 11040 ts_tablet_manager.cc:1918] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:02:02.107945 11040 log.cc:1196] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-2/wal/wals/95bf8a36ecc14009a47ffc8d8f7eb13b
I20260928 16:02:02.108181 11041 ts_tablet_manager.cc:1918] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:02:02.108357 11041 log.cc:1196] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-0/wal/wals/95bf8a36ecc14009a47ffc8d8f7eb13b
I20260928 16:02:02.108731 11040 ts_tablet_manager.cc:1939] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 4f2484f4e25a4351ae70be15e845fb17: Deleting consensus metadata
I20260928 16:02:02.108943 11041 ts_tablet_manager.cc:1939] T 95bf8a36ecc14009a47ffc8d8f7eb13b P b8637a4b055e408494a4dc0abe44f117: Deleting consensus metadata
I20260928 16:02:02.110733 10538 catalog_manager.cc:4841] TS 4f2484f4e25a4351ae70be15e845fb17 (127.22.125.195:38493): tablet 95bf8a36ecc14009a47ffc8d8f7eb13b (table test-table [id=fe2c1d8c11e241948ca90956331fbe56]) successfully deleted
I20260928 16:02:02.110996 10539 catalog_manager.cc:4841] TS b8637a4b055e408494a4dc0abe44f117 (127.22.125.193:42465): tablet 95bf8a36ecc14009a47ffc8d8f7eb13b (table test-table [id=fe2c1d8c11e241948ca90956331fbe56]) successfully deleted
W20260928 16:02:02.111452 10538 catalog_manager.cc:4808] TS 4f2484f4e25a4351ae70be15e845fb17 (127.22.125.195:38493): delete failed for tablet 95bf8a36ecc14009a47ffc8d8f7eb13b because the tablet was not found. No further retry: Not found: Tablet not found: 95bf8a36ecc14009a47ffc8d8f7eb13b
I20260928 16:02:02.113248 11042 tablet_replica.cc:330] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: stopping tablet replica
I20260928 16:02:02.113718 11042 raft_consensus.cc:2229] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 1 LEADER]: Raft consensus shutting down.
I20260928 16:02:02.114358 11042 raft_consensus.cc:2258] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 16:02:02.116492 11042 ts_tablet_manager.cc:1905] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 16:02:02.120247 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:02.100535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.22.125.193:42465)
0928 16:02:02.100726 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:02.100733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:02.112557 (+ 11824us) server_negotiation.cc:407] Connection header received
0928 16:02:02.112757 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:02.112764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:02.112874 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:02.112995 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:02.113494 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.114577 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.116581 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.117228 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.117365 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.118598 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:02.118610 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:02.118627 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:02.119373 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:02.119431 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:02.119457 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:02.119581 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:02.119736 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 16:02:02.119854 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 16:02:02.121006 10733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:02.123143 11042 ts_tablet_manager.cc:1918] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 16:02:02.123315 11042 log.cc:1196] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Deleting WAL directory at /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790611005938867-23031-0/minicluster-data/ts-1/wal/wals/95bf8a36ecc14009a47ffc8d8f7eb13b
I20260928 16:02:02.123904 11042 ts_tablet_manager.cc:1939] T 95bf8a36ecc14009a47ffc8d8f7eb13b P 8406068d10e14ee998e3c75a7ed1ebb7: Deleting consensus metadata
I20260928 16:02:02.125733 10537 catalog_manager.cc:4841] TS 8406068d10e14ee998e3c75a7ed1ebb7 (127.22.125.194:42267): tablet 95bf8a36ecc14009a47ffc8d8f7eb13b (table test-table [id=fe2c1d8c11e241948ca90956331fbe56]) successfully deleted
I20260928 16:02:02.209970 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 10611
I20260928 16:02:02.227074 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 10742
I20260928 16:02:02.234681 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:02.227879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.22.125.194:42267)
0928 16:02:02.228143 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:02.228150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:02.228172 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:02:02.228306 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:02.228311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:02.228430 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:02.228540 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:02.228888 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.230050 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.231238 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.231933 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.232087 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.233070 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:02.233082 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:02.233102 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:02.233907 (+ 805us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:02.233980 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:02.234005 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:02.234110 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:02.234195 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 16:02:02.234296 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260928 16:02:02.235458 10864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:02.346429 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 10742
I20260928 16:02:02.365363 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 10872
I20260928 16:02:02.377745 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:02.366102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.22.125.195:38493)
0928 16:02:02.366280 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:02.366284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:02.366303 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:02:02.366450 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:02.366454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:02.366541 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:02.366637 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:02.366988 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.367785 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.369081 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:02.369458 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:02.369553 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:02.376456 (+ 6903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:02.376464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:02.376479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:02.377045 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:02.377096 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:02.377122 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:02.377213 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:02.377322 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 16:02:02.377414 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:02:02.378373 10994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:02.468940 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 10872
I20260928 16:02:02.485456 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 10521
I20260928 16:02:02.486524 10582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:02.603403 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 10521
2026-09-28T16:02:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2762 ms)
[----------] 6 tests from EncryptionPolicyTest (21314 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:02 dist-test-slave-9cqd krb5kdc[11050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:02 dist-test-slave-9cqd krb5kdc[11050](info): set up 2 sockets
Sep 28 16:02:02 dist-test-slave-9cqd krb5kdc[11050](info): commencing operation
krb5kdc: starting...
W20260928 16:02:03.709050 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:03 dist-test-slave-9cqd krb5kdc[11050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:03.949812 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:04.271997 11063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:04.272318 11063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:04.272414 11063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:04.279294 11063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:04.279393 11063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:04.279438 11063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:04.279491 11063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:04.279556 11063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:04.289227 11063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:04.290454 11063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:04.291702 11063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:04.303022 11068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:04.303068 11069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:04.303112 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:04.303841 11063 server_base.cc:1034] running on GCE node
I20260928 16:02:04.304569 11063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:04.305857 11063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:04.307072 11063 hybrid_clock.cc:648] HybridClock initialized: now 1790611324307034 us; error 49 us; skew 500 ppm
I20260928 16:02:04.307404 11063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:04 dist-test-slave-9cqd krb5kdc[11050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611324, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:04.310388 11063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:04.311762 11063 webserver.cc:460] Webserver started at http://127.0.0.1:34691/ using document root <none> and password file <none>
I20260928 16:02:04.312476 11063 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:04.312605 11063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:04.312966 11063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:04.315747 11063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "57e934b935d648ccb08accbbbd629f0c"
format_stamp: "Formatted at 2026-09-28 16:02:04 on dist-test-slave-9cqd"
server_key: "a8a4eebcb7cac1b4d83796f1df22a262"
server_key_iv: "98f1c456861ba854462df886bc28c5b2"
server_key_version: "encryptionkey@0"
I20260928 16:02:04.316516 11063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "57e934b935d648ccb08accbbbd629f0c"
format_stamp: "Formatted at 2026-09-28 16:02:04 on dist-test-slave-9cqd"
server_key: "a8a4eebcb7cac1b4d83796f1df22a262"
server_key_iv: "98f1c456861ba854462df886bc28c5b2"
server_key_version: "encryptionkey@0"
I20260928 16:02:04.321414 11063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 16:02:04.324952 11078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:04.326179 11063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:02:04.326351 11063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "57e934b935d648ccb08accbbbd629f0c"
format_stamp: "Formatted at 2026-09-28 16:02:04 on dist-test-slave-9cqd"
server_key: "a8a4eebcb7cac1b4d83796f1df22a262"
server_key_iv: "98f1c456861ba854462df886bc28c5b2"
server_key_version: "encryptionkey@0"
I20260928 16:02:04.326534 11063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:04.346678 11063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:04.350389 11063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:04.350663 11063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:04.368247 11063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33443
I20260928 16:02:04.368268 11129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33443 every 8 connection(s)
I20260928 16:02:04.369524 11063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:04.372586 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11063
I20260928 16:02:04.372808 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:04.373492 11130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:04.373660 23031 external_mini_cluster.cc:1428] Setting key 828ec4969de0eb9ef21dbcdbf5088848
I20260928 16:02:04.387743 11130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c: Bootstrap starting.
Sep 28 16:02:04 dist-test-slave-9cqd krb5kdc[11050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:02:04.391196 11130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:04.392340 11130 log.cc:824] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:04.395136 11130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c: No bootstrap required, opened a new log
I20260928 16:02:04.396343 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:04.375613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41715 (local address 127.0.0.1:33443)
0928 16:02:04.376422 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:04.376433 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:04.376476 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:02:04.378243 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:04.378264 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:04.378642 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:04.378943 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:04.380750 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:04.382024 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:04.383845 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:04.384596 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:04.384729 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:04.389028 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:04.389048 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:04.389062 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:04.389105 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:04.392188 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:04.393026 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:04.393033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:04.393039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:04.393205 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:04.393688 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:04.393697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:04.393699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:04.394166 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:04.394478 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:04.394990 (+ 512us) server_negotiation.cc:299] Negotiation successful
0928 16:02:04.395355 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":375,"threads_started":1}
I20260928 16:02:04.400178 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:04.402905 11130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e934b935d648ccb08accbbbd629f0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33443 } }
I20260928 16:02:04.403352 11130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:04.403468 11130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e934b935d648ccb08accbbbd629f0c, State: Initialized, Role: FOLLOWER
I20260928 16:02:04.404129 11130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e934b935d648ccb08accbbbd629f0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33443 } }
I20260928 16:02:04.404381 11130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:04.404531 11130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:04.404748 11130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:04.407124 11130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e934b935d648ccb08accbbbd629f0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33443 } }
I20260928 16:02:04.407619 11130 leader_election.cc:302] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57e934b935d648ccb08accbbbd629f0c; no voters:
I20260928 16:02:04.408278 11130 leader_election.cc:288] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:04.408383 11140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:04.410894 11130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:04.412372 11140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [term 1 LEADER]: Becoming Leader. State: Replica: 57e934b935d648ccb08accbbbd629f0c, State: Running, Role: LEADER
I20260928 16:02:04.413152 11140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e934b935d648ccb08accbbbd629f0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33443 } }
I20260928 16:02:04.417131 11141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57e934b935d648ccb08accbbbd629f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e934b935d648ccb08accbbbd629f0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33443 } } }
I20260928 16:02:04.417606 11141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:04.417121 11142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57e934b935d648ccb08accbbbd629f0c. Latest consensus state: current_term: 1 leader_uuid: "57e934b935d648ccb08accbbbd629f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e934b935d648ccb08accbbbd629f0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33443 } } }
I20260928 16:02:04.419210 11142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:04.421931 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:04.404966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60271 (local address 127.0.0.1:33443)
0928 16:02:04.405147 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:04.405155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:04.405507 (+ 352us) server_negotiation.cc:407] Connection header received
0928 16:02:04.405694 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:04.405699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:04.405777 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:04.405880 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:04.407199 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:04.408097 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:04.409728 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:04.410241 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:04.410363 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:04.412993 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:04.412998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:04.413003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:04.413038 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:04.416678 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:04.417721 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:04.417727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:04.417733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:04.417884 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:04.418605 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:04.418613 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:04.418619 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:04.419151 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:04.419528 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:04.421142 (+ 1614us) server_negotiation.cc:299] Negotiation successful
0928 16:02:04.421395 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 16:02:04.423457 11147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:04.429359 11147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:04.437589 11147 catalog_manager.cc:1347] Generated new cluster ID: 0496618091754f7bad5b37f218bd57b0
I20260928 16:02:04.437695 11147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:04.451279 11147 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:04.453058 11147 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:04.469499 11147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57e934b935d648ccb08accbbbd629f0c: Generated new TSK 0
I20260928 16:02:04.470571 11147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:04.481864 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11063
I20260928 16:02:04.483150 11125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:04.577865 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11063
W20260928 16:02:04.595229 11048 connection.cc:535] client connection to 127.0.0.1:33443 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33443): Connection reset by peer (error 104)
2026-09-28T16:02:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1968 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:04 dist-test-slave-9cqd krb5kdc[11162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:04 dist-test-slave-9cqd krb5kdc[11162](info): set up 2 sockets
Sep 28 16:02:04 dist-test-slave-9cqd krb5kdc[11162](info): commencing operation
krb5kdc: starting...
W20260928 16:02:05.678581 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:05 dist-test-slave-9cqd krb5kdc[11162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:05.923489 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:06.250128 11175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:06.250445 11175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:06.250617 11175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:06.257807 11175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:06.257958 11175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:06.258039 11175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:06.258147 11175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:06.258226 11175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:06.268748 11175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:06.270062 11175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:06.271366 11175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:06.282823 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:06.282790 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:06.282855 11183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:06.283640 11175 server_base.cc:1034] running on GCE node
I20260928 16:02:06.284401 11175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:06.285641 11175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:06.286839 11175 hybrid_clock.cc:648] HybridClock initialized: now 1790611326286795 us; error 32 us; skew 500 ppm
I20260928 16:02:06.287184 11175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:06 dist-test-slave-9cqd krb5kdc[11162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611326, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:06.290699 11175 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:06.292001 11175 webserver.cc:460] Webserver started at http://127.0.0.1:33991/ using document root <none> and password file <none>
I20260928 16:02:06.292658 11175 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:06.292759 11175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:06.292996 11175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:06.295801 11175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be"
format_stamp: "Formatted at 2026-09-28 16:02:06 on dist-test-slave-9cqd"
server_key: "afe2e508c9da584c63aa9597e82367a7"
server_key_iv: "95b16768d7bb1ab9e13bb566e47afa3c"
server_key_version: "encryptionkey@0"
I20260928 16:02:06.296598 11175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be"
format_stamp: "Formatted at 2026-09-28 16:02:06 on dist-test-slave-9cqd"
server_key: "afe2e508c9da584c63aa9597e82367a7"
server_key_iv: "95b16768d7bb1ab9e13bb566e47afa3c"
server_key_version: "encryptionkey@0"
I20260928 16:02:06.301375 11175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260928 16:02:06.304543 11190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:06.305776 11175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260928 16:02:06.305900 11175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be"
format_stamp: "Formatted at 2026-09-28 16:02:06 on dist-test-slave-9cqd"
server_key: "afe2e508c9da584c63aa9597e82367a7"
server_key_iv: "95b16768d7bb1ab9e13bb566e47afa3c"
server_key_version: "encryptionkey@0"
I20260928 16:02:06.306047 11175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:06.328898 11175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:06.332844 11175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:06.333086 11175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:06.351411 11175 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45267
I20260928 16:02:06.351444 11241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45267 every 8 connection(s)
I20260928 16:02:06.352860 11175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:06.356664 11242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:06.357395 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11175
I20260928 16:02:06.357574 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:06.358031 23031 external_mini_cluster.cc:1428] Setting key 85c8cf22e3f072664980bfbdc2094d8d
I20260928 16:02:06.371457 11242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be: Bootstrap starting.
I20260928 16:02:06.373476 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:06.359890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51377 (local address 127.0.0.1:45267)
0928 16:02:06.360738 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:06.360754 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:06.360818 (+ 64us) server_negotiation.cc:407] Connection header received
0928 16:02:06.363374 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:06.363403 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:06.363951 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:06.364442 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:06.364961 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:06.366902 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:06.368274 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:06.369374 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:06.369585 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:06.370689 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:06.370700 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:06.370717 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:06.371426 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:06.371537 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:06.371573 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:06.371725 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:06.371939 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 16:02:06.372134 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":385,"threads_started":1}
I20260928 16:02:06.375088 11242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:06.376322 11242 log.cc:824] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:06.377359 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:06.379019 11242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be: No bootstrap required, opened a new log
I20260928 16:02:06.388537 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:06.381704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58279 (local address 127.0.0.1:45267)
0928 16:02:06.381930 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:06.381939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:06.382331 (+ 392us) server_negotiation.cc:407] Connection header received
0928 16:02:06.382505 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:06.382509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:06.382566 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:06.382643 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:06.383224 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:06.384000 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:06.385899 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:06.386385 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:06.386485 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:06.387745 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:06.387750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:06.387753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:06.387768 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:06.387826 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:06.387841 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:06.387920 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:06.388168 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 16:02:06.388227 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 16:02:06.388514 11242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45267 } }
I20260928 16:02:06.389019 11242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:06.389135 11242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f6f3fe0dfb14f22b7cb6bd65b2c25be, State: Initialized, Role: FOLLOWER
I20260928 16:02:06.389803 11242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45267 } }
I20260928 16:02:06.390082 11242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:06.390223 11242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:06.390431 11242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:06.393060 11242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45267 } }
I20260928 16:02:06.393554 11242 leader_election.cc:302] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f6f3fe0dfb14f22b7cb6bd65b2c25be; no voters:
I20260928 16:02:06.394101 11242 leader_election.cc:288] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:06.394227 11252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:06.394958 11252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [term 1 LEADER]: Becoming Leader. State: Replica: 1f6f3fe0dfb14f22b7cb6bd65b2c25be, State: Running, Role: LEADER
I20260928 16:02:06.395522 11252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45267 } }
I20260928 16:02:06.396445 11242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:06.399382 11253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45267 } } }
I20260928 16:02:06.399870 11253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:06.403223 11254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f6f3fe0dfb14f22b7cb6bd65b2c25be. Latest consensus state: current_term: 1 leader_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6f3fe0dfb14f22b7cb6bd65b2c25be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45267 } } }
I20260928 16:02:06.403538 11254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:06.404161 11261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:06.408707 11261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:06.417085 11261 catalog_manager.cc:1347] Generated new cluster ID: 26dc401e77ce425c858ba3e0aa259b02
I20260928 16:02:06.417196 11261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:06.429234 11261 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:06.431027 11261 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:06.444181 11261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f6f3fe0dfb14f22b7cb6bd65b2c25be: Generated new TSK 0
I20260928 16:02:06.445192 11261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:06.455544 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11175
W20260928 16:02:06.456857 11237 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51377
W20260928 16:02:06.457526 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11175: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:06.457726 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11175
W20260928 16:02:06.471576 11160 connection.cc:535] client connection to 127.0.0.1:45267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45267): Connection reset by peer (error 104)
2026-09-28T16:02:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1876 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:06 dist-test-slave-9cqd krb5kdc[11273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:06 dist-test-slave-9cqd krb5kdc[11273](info): set up 2 sockets
Sep 28 16:02:06 dist-test-slave-9cqd krb5kdc[11273](info): commencing operation
krb5kdc: starting...
W20260928 16:02:07.557991 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:07 dist-test-slave-9cqd krb5kdc[11273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:07.789362 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:08.119552 11286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:08.119882 11286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:08.120074 11286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:08.127286 11286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:08.127425 11286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:08.127521 11286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:08.127625 11286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:08.127701 11286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:08.137647 11286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:08.138902 11286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:08.140338 11286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:08.152161 11291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:08.152258 11292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:08.152426 11294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:08.153239 11286 server_base.cc:1034] running on GCE node
I20260928 16:02:08.153924 11286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:08.155231 11286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:08.156450 11286 hybrid_clock.cc:648] HybridClock initialized: now 1790611328156403 us; error 39 us; skew 500 ppm
I20260928 16:02:08.156793 11286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:08 dist-test-slave-9cqd krb5kdc[11273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611328, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:08.171862 11286 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:08.173599 11286 webserver.cc:460] Webserver started at http://127.0.0.1:36731/ using document root <none> and password file <none>
I20260928 16:02:08.174332 11286 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:08.174480 11286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:08.174793 11286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:08.177855 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "13742ed133ec46b889b7c52e41853b48"
format_stamp: "Formatted at 2026-09-28 16:02:08 on dist-test-slave-9cqd"
server_key: "7ef25fb2a66935bc3649e3cf50b81c57"
server_key_iv: "6bc542761ab55bb27f3935ee7a0e8ea1"
server_key_version: "encryptionkey@0"
I20260928 16:02:08.178634 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "13742ed133ec46b889b7c52e41853b48"
format_stamp: "Formatted at 2026-09-28 16:02:08 on dist-test-slave-9cqd"
server_key: "7ef25fb2a66935bc3649e3cf50b81c57"
server_key_iv: "6bc542761ab55bb27f3935ee7a0e8ea1"
server_key_version: "encryptionkey@0"
I20260928 16:02:08.183820 11286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 16:02:08.187197 11301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:08.188608 11286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:08.188788 11286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "13742ed133ec46b889b7c52e41853b48"
format_stamp: "Formatted at 2026-09-28 16:02:08 on dist-test-slave-9cqd"
server_key: "7ef25fb2a66935bc3649e3cf50b81c57"
server_key_iv: "6bc542761ab55bb27f3935ee7a0e8ea1"
server_key_version: "encryptionkey@0"
I20260928 16:02:08.188987 11286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:08.194725 11286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:08.198597 11286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:08.198886 11286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:08.217042 11286 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33869
I20260928 16:02:08.217092 11353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33869 every 8 connection(s)
I20260928 16:02:08.218431 11286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:08.222545 11354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:08.225553 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11286
I20260928 16:02:08.225726 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:08.226154 23031 external_mini_cluster.cc:1428] Setting key 54d875988c431f961c63c9e57a92367d
I20260928 16:02:08.233721 11357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:08.227905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57341 (local address 127.0.0.1:33869)
0928 16:02:08.228675 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:08.228687 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:08.228725 (+ 38us) server_negotiation.cc:407] Connection header received
0928 16:02:08.230403 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:08.230426 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:08.230774 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:08.231189 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:08.232228 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:08.232234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:08.232246 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:08.232259 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:08.232323 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:08.232342 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:08.232568 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:08.232700 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 16:02:08.232863 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":316,"threads_started":1}
I20260928 16:02:08.234107 11354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48: Bootstrap starting.
I20260928 16:02:08.237236 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:08.237643 11354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:08.238613 11354 log.cc:824] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:08.241442 11354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48: No bootstrap required, opened a new log
I20260928 16:02:08.244220 11357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:08.241781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60987 (local address 127.0.0.1:33869)
0928 16:02:08.241983 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:08.241989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:08.242414 (+ 425us) server_negotiation.cc:407] Connection header received
0928 16:02:08.242536 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:08.242541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:08.242597 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:08.242709 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:08.243557 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:08.243562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:08.243567 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:08.243585 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:08.243632 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:08.243653 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:08.243775 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:08.243850 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 16:02:08.243905 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:02:08.249423 11354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13742ed133ec46b889b7c52e41853b48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260928 16:02:08.249755 11354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:08.249850 11354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13742ed133ec46b889b7c52e41853b48, State: Initialized, Role: FOLLOWER
I20260928 16:02:08.250425 11354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13742ed133ec46b889b7c52e41853b48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260928 16:02:08.250655 11354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:08.250797 11354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:08.250970 11354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:08.253114 11354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13742ed133ec46b889b7c52e41853b48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260928 16:02:08.253603 11354 leader_election.cc:302] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13742ed133ec46b889b7c52e41853b48; no voters:
I20260928 16:02:08.254134 11354 leader_election.cc:288] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:08.254245 11364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:08.255831 11364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [term 1 LEADER]: Becoming Leader. State: Replica: 13742ed133ec46b889b7c52e41853b48, State: Running, Role: LEADER
I20260928 16:02:08.256376 11364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13742ed133ec46b889b7c52e41853b48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260928 16:02:08.256489 11354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:08.258952 11366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13742ed133ec46b889b7c52e41853b48. Latest consensus state: current_term: 1 leader_uuid: "13742ed133ec46b889b7c52e41853b48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13742ed133ec46b889b7c52e41853b48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } } }
I20260928 16:02:08.259346 11366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:08.259883 11365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13742ed133ec46b889b7c52e41853b48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13742ed133ec46b889b7c52e41853b48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } } }
I20260928 16:02:08.260203 11365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:08.261211 11371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:08.266624 11371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:08.275578 11371 catalog_manager.cc:1347] Generated new cluster ID: efe5ce22a0f44964a7e13143d90a2bbf
I20260928 16:02:08.275691 11371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:08.294792 11371 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:08.295244 11371 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:08.304441 11371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13742ed133ec46b889b7c52e41853b48: Generated new TSK 0
I20260928 16:02:08.305326 11371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:08.318248 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11286
W20260928 16:02:08.319233 11349 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57341
W20260928 16:02:08.319882 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11286: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:08.320016 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11286
2026-09-28T16:02:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1865 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:08 dist-test-slave-9cqd krb5kdc[11385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:08 dist-test-slave-9cqd krb5kdc[11385](info): set up 2 sockets
Sep 28 16:02:08 dist-test-slave-9cqd krb5kdc[11385](info): commencing operation
krb5kdc: starting...
W20260928 16:02:09.418238 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:09 dist-test-slave-9cqd krb5kdc[11385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:09.664968 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:09.991520 11398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:09.991878 11398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:09.992025 11398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:09.999176 11398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:09.999315 11398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:09.999406 11398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:09.999505 11398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:09.999575 11398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:10.009313 11398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:10.010562 11398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:10.011891 11398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:10.024137 11403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:10.024178 11404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:10.024216 11406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:10.025122 11398 server_base.cc:1034] running on GCE node
I20260928 16:02:10.025893 11398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:10.027345 11398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:10.028596 11398 hybrid_clock.cc:648] HybridClock initialized: now 1790611330028534 us; error 47 us; skew 500 ppm
I20260928 16:02:10.028966 11398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:10 dist-test-slave-9cqd krb5kdc[11385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611330, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:10.032102 11398 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:10.033610 11398 webserver.cc:460] Webserver started at http://127.0.0.1:41757/ using document root <none> and password file <none>
I20260928 16:02:10.034272 11398 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:10.034427 11398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:10.034723 11398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:10.037997 11398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "2ed2ff997cbb4fc5acce2a47651a47c6"
format_stamp: "Formatted at 2026-09-28 16:02:10 on dist-test-slave-9cqd"
server_key: "526d6cabbda503d712d724fd7b36497a"
server_key_iv: "930e75c4b4f6c2896c2cdb7bc1433fd5"
server_key_version: "encryptionkey@0"
I20260928 16:02:10.038812 11398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "2ed2ff997cbb4fc5acce2a47651a47c6"
format_stamp: "Formatted at 2026-09-28 16:02:10 on dist-test-slave-9cqd"
server_key: "526d6cabbda503d712d724fd7b36497a"
server_key_iv: "930e75c4b4f6c2896c2cdb7bc1433fd5"
server_key_version: "encryptionkey@0"
I20260928 16:02:10.044335 11398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 16:02:10.047770 11413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:10.049032 11398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:10.049247 11398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "2ed2ff997cbb4fc5acce2a47651a47c6"
format_stamp: "Formatted at 2026-09-28 16:02:10 on dist-test-slave-9cqd"
server_key: "526d6cabbda503d712d724fd7b36497a"
server_key_iv: "930e75c4b4f6c2896c2cdb7bc1433fd5"
server_key_version: "encryptionkey@0"
I20260928 16:02:10.049441 11398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:10.073730 11398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:10.078482 11398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:10.078807 11398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:10.097958 11398 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38095
I20260928 16:02:10.098052 11464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38095 every 8 connection(s)
I20260928 16:02:10.099463 11398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:10.103200 11465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:10.109884 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11398
I20260928 16:02:10.110112 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:10.110682 23031 external_mini_cluster.cc:1428] Setting key 78474681978f29fd38fd0ed7511c6350
I20260928 16:02:10.114707 11465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6: Bootstrap starting.
I20260928 16:02:10.117841 11465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:10.118885 11465 log.cc:824] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:10.121553 11465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6: No bootstrap required, opened a new log
Sep 28 16:02:10 dist-test-slave-9cqd krb5kdc[11385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:02:10.130432 11465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38095 } }
I20260928 16:02:10.130861 11465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:10.130990 11465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed2ff997cbb4fc5acce2a47651a47c6, State: Initialized, Role: FOLLOWER
I20260928 16:02:10.131577 11465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38095 } }
I20260928 16:02:10.131839 11465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:10.131994 11465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:10.132238 11465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:10.133958 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:10.112622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51105 (local address 127.0.0.1:38095)
0928 16:02:10.113375 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:10.113387 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:10.113426 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:02:10.115130 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:10.115152 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:10.115535 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:10.115926 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:10.117571 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:10.118931 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:10.120355 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:10.121034 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:10.121277 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:10.125830 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:10.125837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:10.125851 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:10.125884 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:10.129505 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:10.130357 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:10.130363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:10.130368 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:10.130533 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:10.131001 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:10.131007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:10.131010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:10.131684 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:10.132092 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:10.132541 (+ 449us) server_negotiation.cc:299] Negotiation successful
0928 16:02:10.132937 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":349,"threads_started":1}
I20260928 16:02:10.134846 11465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38095 } }
I20260928 16:02:10.135527 11465 leader_election.cc:302] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ed2ff997cbb4fc5acce2a47651a47c6; no voters:
I20260928 16:02:10.136224 11465 leader_election.cc:288] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:10.136345 11470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:10.137823 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:10.139065 11465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:10.143690 11470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ed2ff997cbb4fc5acce2a47651a47c6, State: Running, Role: LEADER
I20260928 16:02:10.144362 11470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38095 } }
I20260928 16:02:10.147769 11471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38095 } } }
I20260928 16:02:10.148288 11471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:10.154829 11472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ed2ff997cbb4fc5acce2a47651a47c6. Latest consensus state: current_term: 1 leader_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed2ff997cbb4fc5acce2a47651a47c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38095 } } }
I20260928 16:02:10.155869 11481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:10.158300 11472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:10.161633 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:10.142719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33711 (local address 127.0.0.1:38095)
0928 16:02:10.142885 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:10.142893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:10.142911 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:02:10.142958 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:10.142963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:10.143030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:10.143143 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:10.144846 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:10.145674 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:10.147288 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:10.147996 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:10.148134 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:10.152238 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:10.152248 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:10.152255 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:10.152305 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:10.157260 (+ 4955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:10.158334 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:10.158343 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:10.158350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:10.158526 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:10.159045 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:10.159050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:10.159053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:10.159517 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:10.159761 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:10.160960 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0928 16:02:10.161212 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:02:10.163267 11481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:10.173513 11481 catalog_manager.cc:1347] Generated new cluster ID: 9013bb64ccfb43578b6cc2ec89f3f6d9
I20260928 16:02:10.173638 11481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:10.194125 11481 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:10.195547 11481 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:10.203433 11481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ed2ff997cbb4fc5acce2a47651a47c6: Generated new TSK 0
I20260928 16:02:10.204282 11481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:10.209755 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11398
I20260928 16:02:10.210762 11460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:10.300861 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11398
2026-09-28T16:02:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1981 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:10 dist-test-slave-9cqd krb5kdc[11498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:10 dist-test-slave-9cqd krb5kdc[11498](info): set up 2 sockets
Sep 28 16:02:10 dist-test-slave-9cqd krb5kdc[11498](info): commencing operation
krb5kdc: starting...
W20260928 16:02:11.398161 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:11 dist-test-slave-9cqd krb5kdc[11498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:11.638988 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:11.963438 11511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:11.963698 11511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:11.963855 11511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:11.971032 11511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:11.971174 11511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:11.971283 11511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:11.971383 11511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:11.971467 11511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:11.981189 11511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:11.982412 11511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:11.983687 11511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:11.995394 11516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:11.995505 11517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:11.995553 11519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:11.996513 11511 server_base.cc:1034] running on GCE node
I20260928 16:02:11.997248 11511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:11.998612 11511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:11.999835 11511 hybrid_clock.cc:648] HybridClock initialized: now 1790611331999796 us; error 58 us; skew 500 ppm
I20260928 16:02:12.000239 11511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:11 dist-test-slave-9cqd krb5kdc[11498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611331, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:12.003471 11511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:12.004895 11511 webserver.cc:460] Webserver started at http://127.0.0.1:43973/ using document root <none> and password file <none>
I20260928 16:02:12.005548 11511 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:12.005702 11511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:12.005996 11511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:12.008899 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "45e3a91b282743ed89747c1bd8cc4aad"
format_stamp: "Formatted at 2026-09-28 16:02:12 on dist-test-slave-9cqd"
server_key: "59ba3b367f41573f2dc38e775b117a6a"
server_key_iv: "29a79f82e271b42551f2f891c0833054"
server_key_version: "encryptionkey@0"
I20260928 16:02:12.009672 11511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "45e3a91b282743ed89747c1bd8cc4aad"
format_stamp: "Formatted at 2026-09-28 16:02:12 on dist-test-slave-9cqd"
server_key: "59ba3b367f41573f2dc38e775b117a6a"
server_key_iv: "29a79f82e271b42551f2f891c0833054"
server_key_version: "encryptionkey@0"
I20260928 16:02:12.014569 11511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260928 16:02:12.017836 11526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:12.019016 11511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 16:02:12.019207 11511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "45e3a91b282743ed89747c1bd8cc4aad"
format_stamp: "Formatted at 2026-09-28 16:02:12 on dist-test-slave-9cqd"
server_key: "59ba3b367f41573f2dc38e775b117a6a"
server_key_iv: "29a79f82e271b42551f2f891c0833054"
server_key_version: "encryptionkey@0"
I20260928 16:02:12.019403 11511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:12.044847 11511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:12.049199 11511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:12.049484 11511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:12.067838 11511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42499
I20260928 16:02:12.067876 11577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42499 every 8 connection(s)
I20260928 16:02:12.069181 11511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:12.071746 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11511
I20260928 16:02:12.071942 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:12.072510 23031 external_mini_cluster.cc:1428] Setting key 7390111c556b7d1507e9a45d713b5040
I20260928 16:02:12.073323 11578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:12.085412 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:12.074185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57351 (local address 127.0.0.1:42499)
0928 16:02:12.075002 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:12.075032 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:12.075070 (+ 38us) server_negotiation.cc:407] Connection header received
0928 16:02:12.076639 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:12.076659 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:12.077029 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:12.077341 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:12.077898 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:12.079264 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:12.081106 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:12.081740 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:12.081867 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:12.083128 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:12.083136 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:12.083148 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:12.083698 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:12.083771 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:12.083802 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:12.083886 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:12.084202 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 16:02:12.084491 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":362,"threads_started":1}
I20260928 16:02:12.086582 11578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad: Bootstrap starting.
I20260928 16:02:12.088887 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:12.089993 11578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:12.091161 11578 log.cc:824] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:12.093879 11578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad: No bootstrap required, opened a new log
I20260928 16:02:12.101150 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:12.094379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51359 (local address 127.0.0.1:42499)
0928 16:02:12.094553 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:12.094559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:12.095145 (+ 586us) server_negotiation.cc:407] Connection header received
0928 16:02:12.095359 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:12.095363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:12.095419 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:12.095491 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:12.096135 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:12.096872 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:12.098700 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:12.099168 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:12.099258 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:12.100459 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:12.100464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:12.100467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:12.100482 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:12.100520 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:12.100534 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:12.100608 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:12.100750 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 16:02:12.100818 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:02:12.103911 11578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3a91b282743ed89747c1bd8cc4aad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260928 16:02:12.104350 11578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:12.104460 11578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e3a91b282743ed89747c1bd8cc4aad, State: Initialized, Role: FOLLOWER
I20260928 16:02:12.105005 11578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3a91b282743ed89747c1bd8cc4aad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260928 16:02:12.105242 11578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:12.105394 11578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:12.105609 11578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:12.107771 11578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3a91b282743ed89747c1bd8cc4aad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260928 16:02:12.108309 11578 leader_election.cc:302] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45e3a91b282743ed89747c1bd8cc4aad; no voters:
I20260928 16:02:12.108944 11578 leader_election.cc:288] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:12.109048 11588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:12.109831 11588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [term 1 LEADER]: Becoming Leader. State: Replica: 45e3a91b282743ed89747c1bd8cc4aad, State: Running, Role: LEADER
I20260928 16:02:12.110416 11588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3a91b282743ed89747c1bd8cc4aad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260928 16:02:12.111156 11578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:12.113498 11590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45e3a91b282743ed89747c1bd8cc4aad. Latest consensus state: current_term: 1 leader_uuid: "45e3a91b282743ed89747c1bd8cc4aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3a91b282743ed89747c1bd8cc4aad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } } }
I20260928 16:02:12.113677 11589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45e3a91b282743ed89747c1bd8cc4aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3a91b282743ed89747c1bd8cc4aad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } } }
I20260928 16:02:12.114216 11589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:12.114264 11590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:12.115832 11597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:12.121213 11597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:12.130904 11597 catalog_manager.cc:1347] Generated new cluster ID: 62437f58b8cb4e5f9afaf24965b06656
I20260928 16:02:12.131024 11597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:12.144342 11597 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:12.145602 11597 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:12.154124 11597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45e3a91b282743ed89747c1bd8cc4aad: Generated new TSK 0
I20260928 16:02:12.154932 11597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:12.166301 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11511
W20260928 16:02:12.167577 11573 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57351
W20260928 16:02:12.168319 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:12.168476 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11511
2026-09-28T16:02:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1866 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:12 dist-test-slave-9cqd krb5kdc[11609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:12 dist-test-slave-9cqd krb5kdc[11609](info): set up 2 sockets
Sep 28 16:02:12 dist-test-slave-9cqd krb5kdc[11609](info): commencing operation
krb5kdc: starting...
W20260928 16:02:13.262048 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:13 dist-test-slave-9cqd krb5kdc[11609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:13.498927 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33317
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:13.837987 11623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:13.838323 11623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:13.838497 11623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:13.845948 11623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:13.846135 11623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:13.846252 11623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:13.846361 11623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:13.846457 11623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:13.856139 11623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:13.857431 11623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:13.858806 11623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:13.870600 11628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:13.870645 11631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:13.870671 11629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:13.871320 11623 server_base.cc:1034] running on GCE node
I20260928 16:02:13.872025 11623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:13.873431 11623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:13.874658 11623 hybrid_clock.cc:648] HybridClock initialized: now 1790611333874619 us; error 50 us; skew 500 ppm
I20260928 16:02:13.875002 11623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:13 dist-test-slave-9cqd krb5kdc[11609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611333, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:13.878141 11623 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:13.879590 11623 webserver.cc:460] Webserver started at http://127.0.0.1:40117/ using document root <none> and password file <none>
I20260928 16:02:13.880301 11623 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:13.880442 11623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:13.880770 11623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:13.883700 11623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "b4fbf2671cc64d19bf2638ac8f64cb60"
format_stamp: "Formatted at 2026-09-28 16:02:13 on dist-test-slave-9cqd"
server_key: "255eee23368f6807cde5f80329c1ab1e"
server_key_iv: "eccdf9e8587d1ca41be9dfbd5491f206"
server_key_version: "encryptionkey@0"
I20260928 16:02:13.884492 11623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "b4fbf2671cc64d19bf2638ac8f64cb60"
format_stamp: "Formatted at 2026-09-28 16:02:13 on dist-test-slave-9cqd"
server_key: "255eee23368f6807cde5f80329c1ab1e"
server_key_iv: "eccdf9e8587d1ca41be9dfbd5491f206"
server_key_version: "encryptionkey@0"
I20260928 16:02:13.889510 11623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260928 16:02:13.892872 11638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:13.894136 11623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:02:13.894307 11623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "b4fbf2671cc64d19bf2638ac8f64cb60"
format_stamp: "Formatted at 2026-09-28 16:02:13 on dist-test-slave-9cqd"
server_key: "255eee23368f6807cde5f80329c1ab1e"
server_key_iv: "eccdf9e8587d1ca41be9dfbd5491f206"
server_key_version: "encryptionkey@0"
I20260928 16:02:13.894495 11623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:13.900307 11623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:13.904381 11623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:13.904686 11623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:13.923034 11623 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33317
I20260928 16:02:13.923081 11689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33317 every 8 connection(s)
I20260928 16:02:13.924409 11623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:13.927969 11690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:13.931707 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11623
I20260928 16:02:13.931898 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:13.932516 23031 external_mini_cluster.cc:1428] Setting key 0f74c4091ca5422de7cfd22903eb8134
I20260928 16:02:13.939868 11693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:13.934241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44689 (local address 127.0.0.1:33317)
0928 16:02:13.935001 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:13.935012 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:13.935051 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:02:13.936656 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:13.936677 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:13.937014 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:13.937321 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:13.938339 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:13.938348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:13.938360 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:13.938383 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:13.938434 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:13.938458 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:13.938622 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:13.938742 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 16:02:13.938958 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":348,"threads_started":1}
I20260928 16:02:13.940546 11690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60: Bootstrap starting.
I20260928 16:02:13.943305 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:13.943639 11690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:13.944837 11690 log.cc:824] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:13.947981 11690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60: No bootstrap required, opened a new log
I20260928 16:02:13.951269 11693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:13.948642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49233 (local address 127.0.0.1:33317)
0928 16:02:13.948859 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:13.948868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:13.949364 (+ 496us) server_negotiation.cc:407] Connection header received
0928 16:02:13.949560 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:13.949564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:13.949609 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:13.949720 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:13.950617 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:13.950631 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:13.950637 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:13.950652 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:13.950690 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:13.950704 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:13.950845 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:13.950964 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 16:02:13.951021 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 16:02:13.955847 11690 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33317 } }
I20260928 16:02:13.956358 11690 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:13.956522 11690 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fbf2671cc64d19bf2638ac8f64cb60, State: Initialized, Role: FOLLOWER
I20260928 16:02:13.957139 11690 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33317 } }
I20260928 16:02:13.957381 11690 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:13.957532 11690 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:13.957717 11690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:13.959832 11690 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33317 } }
I20260928 16:02:13.960336 11690 leader_election.cc:302] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4fbf2671cc64d19bf2638ac8f64cb60; no voters:
I20260928 16:02:13.960896 11690 leader_election.cc:288] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:13.961040 11700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:13.961700 11700 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [term 1 LEADER]: Becoming Leader. State: Replica: b4fbf2671cc64d19bf2638ac8f64cb60, State: Running, Role: LEADER
I20260928 16:02:13.962282 11700 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33317 } }
I20260928 16:02:13.962942 11690 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:13.964982 11701 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33317 } } }
I20260928 16:02:13.965377 11701 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:13.965158 11702 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4fbf2671cc64d19bf2638ac8f64cb60. Latest consensus state: current_term: 1 leader_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4fbf2671cc64d19bf2638ac8f64cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33317 } } }
I20260928 16:02:13.965674 11702 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:13.966631 11707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:13.972334 11707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:13.980279 11707 catalog_manager.cc:1347] Generated new cluster ID: d262b21991134ddcb0f7d251f50e5c6a
I20260928 16:02:13.980391 11707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:13.998651 11707 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:13.999096 11707 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:14.005882 11707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4fbf2671cc64d19bf2638ac8f64cb60: Generated new TSK 0
I20260928 16:02:14.006703 11707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:14.013410 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11623
W20260928 16:02:14.014441 11685 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44689
W20260928 16:02:14.015106 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11623: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:14.015245 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11623
2026-09-28T16:02:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1846 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:14 dist-test-slave-9cqd krb5kdc[11721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:14 dist-test-slave-9cqd krb5kdc[11721](info): set up 2 sockets
Sep 28 16:02:14 dist-test-slave-9cqd krb5kdc[11721](info): commencing operation
krb5kdc: starting...
W20260928 16:02:15.110399 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:15 dist-test-slave-9cqd krb5kdc[11721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:15.345160 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42315
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:15.669287 11734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:15.669601 11734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:15.669768 11734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:15.676699 11734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:15.676842 11734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:15.676949 11734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:15.677057 11734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:15.677150 11734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:15.688032 11734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:15.689648 11734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:15.690941 11734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:15.702346 11739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:15.702579 11742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:15.702406 11740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:15.703610 11734 server_base.cc:1034] running on GCE node
I20260928 16:02:15.704486 11734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:15.705842 11734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:15.707072 11734 hybrid_clock.cc:648] HybridClock initialized: now 1790611335707035 us; error 48 us; skew 500 ppm
I20260928 16:02:15.707446 11734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:15 dist-test-slave-9cqd krb5kdc[11721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611335, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:15.710546 11734 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:15.712109 11734 webserver.cc:460] Webserver started at http://127.0.0.1:42603/ using document root <none> and password file <none>
I20260928 16:02:15.712751 11734 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:15.712886 11734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:15.713197 11734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:15.716197 11734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "b993bb957be046cd90937a29245115e2"
format_stamp: "Formatted at 2026-09-28 16:02:15 on dist-test-slave-9cqd"
server_key: "92d2559661777fd84fa56f5f05a7b759"
server_key_iv: "c937b82bf6f7795ea9a2a0b7439156bd"
server_key_version: "encryptionkey@0"
I20260928 16:02:15.716985 11734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "b993bb957be046cd90937a29245115e2"
format_stamp: "Formatted at 2026-09-28 16:02:15 on dist-test-slave-9cqd"
server_key: "92d2559661777fd84fa56f5f05a7b759"
server_key_iv: "c937b82bf6f7795ea9a2a0b7439156bd"
server_key_version: "encryptionkey@0"
I20260928 16:02:15.721886 11734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:02:15.725174 11749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:15.726467 11734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260928 16:02:15.726648 11734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "b993bb957be046cd90937a29245115e2"
format_stamp: "Formatted at 2026-09-28 16:02:15 on dist-test-slave-9cqd"
server_key: "92d2559661777fd84fa56f5f05a7b759"
server_key_iv: "c937b82bf6f7795ea9a2a0b7439156bd"
server_key_version: "encryptionkey@0"
I20260928 16:02:15.726807 11734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:15.750142 11734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:15.753988 11734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:15.754266 11734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:15.772531 11734 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42315
I20260928 16:02:15.772567 11800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42315 every 8 connection(s)
I20260928 16:02:15.773816 11734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:15.776793 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11734
I20260928 16:02:15.776970 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:15.777405 23031 external_mini_cluster.cc:1428] Setting key b8f87fbc4b5d55f2658f45752f8d9d73
I20260928 16:02:15.777505 11801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:15.789363 11801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2: Bootstrap starting.
Sep 28 16:02:15 dist-test-slave-9cqd krb5kdc[11721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:02:15.792640 11801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:15.793808 11801 log.cc:824] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:15.796541 11801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2: No bootstrap required, opened a new log
I20260928 16:02:15.799433 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:15.779154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:39763 (local address 127.0.0.1:42315)
0928 16:02:15.779833 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:15.779847 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:15.779887 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:02:15.782057 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:15.782077 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:15.782456 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:15.782847 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:15.784356 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:15.785583 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:15.786836 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:15.787550 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:15.787761 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:15.792146 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:15.792155 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:15.792171 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:15.792203 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:15.795333 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:15.796157 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:15.796164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:15.796169 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:15.796338 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:15.796832 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:15.796836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:15.796838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:15.797320 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:15.797659 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:15.798178 (+ 519us) server_negotiation.cc:299] Negotiation successful
0928 16:02:15.798458 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":293,"threads_started":1}
I20260928 16:02:15.803270 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:15.804522 11801 raft_consensus.cc:348] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b993bb957be046cd90937a29245115e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42315 } }
I20260928 16:02:15.804875 11801 raft_consensus.cc:374] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:15.804989 11801 raft_consensus.cc:729] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b993bb957be046cd90937a29245115e2, State: Initialized, Role: FOLLOWER
I20260928 16:02:15.805560 11801 consensus_queue.cc:260] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b993bb957be046cd90937a29245115e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42315 } }
I20260928 16:02:15.805805 11801 raft_consensus.cc:388] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:15.805967 11801 raft_consensus.cc:482] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:15.806161 11801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:15.808350 11801 raft_consensus.cc:504] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b993bb957be046cd90937a29245115e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42315 } }
I20260928 16:02:15.808876 11801 leader_election.cc:302] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b993bb957be046cd90937a29245115e2; no voters:
I20260928 16:02:15.809535 11801 leader_election.cc:288] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:15.809677 11810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:15.810664 11810 raft_consensus.cc:686] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [term 1 LEADER]: Becoming Leader. State: Replica: b993bb957be046cd90937a29245115e2, State: Running, Role: LEADER
I20260928 16:02:15.811390 11810 consensus_queue.cc:237] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b993bb957be046cd90937a29245115e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42315 } }
I20260928 16:02:15.813691 11811 sys_catalog.cc:455] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b993bb957be046cd90937a29245115e2. Latest consensus state: current_term: 1 leader_uuid: "b993bb957be046cd90937a29245115e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b993bb957be046cd90937a29245115e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42315 } } }
I20260928 16:02:15.815219 11801 sys_catalog.cc:564] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:15.814755 11813 sys_catalog.cc:455] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b993bb957be046cd90937a29245115e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b993bb957be046cd90937a29245115e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42315 } } }
I20260928 16:02:15.815217 11811 sys_catalog.cc:458] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:15.815828 11813 sys_catalog.cc:458] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:15.816701 11815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:15.834339 11815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:15.839274 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:15.810046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:48627 (local address 127.0.0.1:42315)
0928 16:02:15.810271 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:15.810278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:15.817096 (+ 6818us) server_negotiation.cc:407] Connection header received
0928 16:02:15.817310 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:15.817316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:15.817409 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:15.817535 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:15.820143 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:15.821660 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:15.824141 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:15.825080 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:15.825203 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:15.829542 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:15.829551 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:15.829559 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:15.829610 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:15.834553 (+ 4943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:15.835841 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:15.835849 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:15.835856 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:15.836065 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:15.836782 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:15.836805 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:15.836811 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:15.837374 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:15.837767 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:15.838634 (+ 867us) server_negotiation.cc:299] Negotiation successful
0928 16:02:15.838876 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 16:02:15.848186 11815 catalog_manager.cc:1347] Generated new cluster ID: 6c591d9782634740acefcc1a73b519a2
I20260928 16:02:15.848325 11815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:15.862264 11815 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:15.863427 11815 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:15.875567 11815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b993bb957be046cd90937a29245115e2: Generated new TSK 0
I20260928 16:02:15.876399 11815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:15.887223 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11734
I20260928 16:02:15.889051 11796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:15.979364 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11734
W20260928 16:02:15.996443 11719 connection.cc:535] client connection to 127.0.0.1:42315 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42315): Connection reset by peer (error 104)
2026-09-28T16:02:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1964 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:16 dist-test-slave-9cqd krb5kdc[11833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:16 dist-test-slave-9cqd krb5kdc[11833](info): set up 2 sockets
Sep 28 16:02:16 dist-test-slave-9cqd krb5kdc[11833](info): commencing operation
krb5kdc: starting...
W20260928 16:02:17.078210 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:17 dist-test-slave-9cqd krb5kdc[11833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:17.318856 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:17.647161 11846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:17.647418 11846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:17.647589 11846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:17.654510 11846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:17.654603 11846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:17.654646 11846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:17.654726 11846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:17.654789 11846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:17.664090 11846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:17.665292 11846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:17.666594 11846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:17.678107 11851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:17.678157 11854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:17.678184 11852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:17.679073 11846 server_base.cc:1034] running on GCE node
I20260928 16:02:17.679797 11846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:17.681213 11846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:17.682433 11846 hybrid_clock.cc:648] HybridClock initialized: now 1790611337682382 us; error 46 us; skew 500 ppm
I20260928 16:02:17.682780 11846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:17 dist-test-slave-9cqd krb5kdc[11833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611337, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:17.686326 11846 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:17.688649 11846 webserver.cc:460] Webserver started at http://127.0.0.1:45971/ using document root <none> and password file <none>
I20260928 16:02:17.689476 11846 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:17.689610 11846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:17.689904 11846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:17.692828 11846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "6953e920237346099f363436050a1992"
format_stamp: "Formatted at 2026-09-28 16:02:17 on dist-test-slave-9cqd"
server_key: "c17b58a0fddc1fcb7fdbd751cbf1fabe"
server_key_iv: "21323fcda0ce70ef02bb1b595d257705"
server_key_version: "encryptionkey@0"
I20260928 16:02:17.693573 11846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "6953e920237346099f363436050a1992"
format_stamp: "Formatted at 2026-09-28 16:02:17 on dist-test-slave-9cqd"
server_key: "c17b58a0fddc1fcb7fdbd751cbf1fabe"
server_key_iv: "21323fcda0ce70ef02bb1b595d257705"
server_key_version: "encryptionkey@0"
I20260928 16:02:17.698604 11846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:02:17.702037 11861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:17.703298 11846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:02:17.703491 11846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "6953e920237346099f363436050a1992"
format_stamp: "Formatted at 2026-09-28 16:02:17 on dist-test-slave-9cqd"
server_key: "c17b58a0fddc1fcb7fdbd751cbf1fabe"
server_key_iv: "21323fcda0ce70ef02bb1b595d257705"
server_key_version: "encryptionkey@0"
I20260928 16:02:17.703687 11846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:17.731406 11846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:17.735389 11846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:17.735673 11846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:17.753824 11846 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35089
I20260928 16:02:17.753861 11912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35089 every 8 connection(s)
I20260928 16:02:17.755141 11846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:17.758790 11913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:17.761533 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11846
I20260928 16:02:17.761722 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:17.762202 23031 external_mini_cluster.cc:1428] Setting key eb51728ad7f635e155f1fd7be1dbd094
I20260928 16:02:17.770560 11913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992: Bootstrap starting.
I20260928 16:02:17.774008 11913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:17.775148 11913 log.cc:824] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:17.775283 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:17.763982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:34679 (local address 127.0.0.1:35089)
0928 16:02:17.764866 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:17.764876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:17.764913 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:02:17.766530 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:17.766550 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:17.766920 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:17.767231 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:17.767807 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:17.769141 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:17.771074 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:17.771728 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:17.771880 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:17.773166 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:17.773172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:17.773184 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:17.773647 (+ 463us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:17.773716 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:17.773739 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:17.773861 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:17.774245 (+ 384us) server_negotiation.cc:299] Negotiation successful
0928 16:02:17.774380 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":380,"threads_started":1}
I20260928 16:02:17.778177 11913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992: No bootstrap required, opened a new log
I20260928 16:02:17.779030 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:17.786459 11913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6953e920237346099f363436050a1992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260928 16:02:17.786831 11913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:17.786935 11913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6953e920237346099f363436050a1992, State: Initialized, Role: FOLLOWER
I20260928 16:02:17.787487 11913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6953e920237346099f363436050a1992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260928 16:02:17.787794 11913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:17.787947 11913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:17.788165 11913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:17.790289 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:17.783467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:57607 (local address 127.0.0.1:35089)
0928 16:02:17.783652 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:17.783657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:17.784092 (+ 435us) server_negotiation.cc:407] Connection header received
0928 16:02:17.784274 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:17.784280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:17.784367 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:17.784467 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:17.784927 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:17.786084 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:17.787613 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:17.788344 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:17.788486 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:17.789525 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:17.789532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:17.789537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:17.789563 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:17.789625 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:17.789644 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:17.789764 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:17.789922 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 16:02:17.789979 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 16:02:17.790946 11913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6953e920237346099f363436050a1992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260928 16:02:17.791523 11913 leader_election.cc:302] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6953e920237346099f363436050a1992; no voters:
I20260928 16:02:17.792152 11913 leader_election.cc:288] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:17.792243 11923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:17.792963 11923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [term 1 LEADER]: Becoming Leader. State: Replica: 6953e920237346099f363436050a1992, State: Running, Role: LEADER
I20260928 16:02:17.793550 11923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6953e920237346099f363436050a1992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260928 16:02:17.794250 11913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:17.796509 11924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6953e920237346099f363436050a1992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6953e920237346099f363436050a1992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } } }
I20260928 16:02:17.796463 11925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6953e920237346099f363436050a1992. Latest consensus state: current_term: 1 leader_uuid: "6953e920237346099f363436050a1992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6953e920237346099f363436050a1992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } } }
I20260928 16:02:17.797056 11925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:17.798753 11924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:17.799528 11929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:17.804411 11929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:17.813035 11929 catalog_manager.cc:1347] Generated new cluster ID: 5e136bdb5c5644ac93ab3063c703a650
I20260928 16:02:17.813197 11929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:17.832839 11929 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:17.834060 11929 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:17.839139 11929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6953e920237346099f363436050a1992: Generated new TSK 0
I20260928 16:02:17.839918 11929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:17.848008 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11846
W20260928 16:02:17.849157 11908 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:34679
W20260928 16:02:17.849925 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11846: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:17.850075 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11846
W20260928 16:02:17.867209 11831 connection.cc:535] client connection to 127.0.0.1:35089 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35089): Connection reset by peer (error 104)
2026-09-28T16:02:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1870 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:17 dist-test-slave-9cqd krb5kdc[11944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:17 dist-test-slave-9cqd krb5kdc[11944](info): set up 2 sockets
Sep 28 16:02:17 dist-test-slave-9cqd krb5kdc[11944](info): commencing operation
krb5kdc: starting...
W20260928 16:02:18.954111 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:19 dist-test-slave-9cqd krb5kdc[11944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:19.195164 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46471
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:19.518891 11957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:19.519155 11957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:19.519366 11957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:19.526328 11957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:19.526443 11957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:19.526530 11957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:19.526628 11957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:19.526708 11957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:19.536334 11957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:19.537585 11957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:19.538869 11957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:19.550856 11962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:19.550900 11965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:19.550943 11963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:19.551863 11957 server_base.cc:1034] running on GCE node
I20260928 16:02:19.552644 11957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:19.554004 11957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:19.555227 11957 hybrid_clock.cc:648] HybridClock initialized: now 1790611339555191 us; error 49 us; skew 500 ppm
I20260928 16:02:19.555568 11957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:19 dist-test-slave-9cqd krb5kdc[11944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611339, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:19.565142 11957 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:19.566601 11957 webserver.cc:460] Webserver started at http://127.0.0.1:40465/ using document root <none> and password file <none>
I20260928 16:02:19.567233 11957 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:19.567334 11957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:19.567579 11957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:19.570506 11957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "09ac9f7510cc4be9a39aaa51f5d00b78"
format_stamp: "Formatted at 2026-09-28 16:02:19 on dist-test-slave-9cqd"
server_key: "50c74a1edf9d719962e5ffa1475482fb"
server_key_iv: "5a297798d9ba275b51bdcd89b8d22dd4"
server_key_version: "encryptionkey@0"
I20260928 16:02:19.571165 11957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "09ac9f7510cc4be9a39aaa51f5d00b78"
format_stamp: "Formatted at 2026-09-28 16:02:19 on dist-test-slave-9cqd"
server_key: "50c74a1edf9d719962e5ffa1475482fb"
server_key_iv: "5a297798d9ba275b51bdcd89b8d22dd4"
server_key_version: "encryptionkey@0"
I20260928 16:02:19.576123 11957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260928 16:02:19.579334 11972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:19.580626 11957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 16:02:19.580803 11957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "09ac9f7510cc4be9a39aaa51f5d00b78"
format_stamp: "Formatted at 2026-09-28 16:02:19 on dist-test-slave-9cqd"
server_key: "50c74a1edf9d719962e5ffa1475482fb"
server_key_iv: "5a297798d9ba275b51bdcd89b8d22dd4"
server_key_version: "encryptionkey@0"
I20260928 16:02:19.580974 11957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:19.586428 11957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:19.590528 11957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:19.590816 11957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:19.608546 11957 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46471
I20260928 16:02:19.608618 12023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46471 every 8 connection(s)
I20260928 16:02:19.609805 11957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:19.613287 12024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:19.617269 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 11957
I20260928 16:02:19.617444 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:19.617988 23031 external_mini_cluster.cc:1428] Setting key 7aed6034f5b75bb348cfd58b6d7ea8d1
I20260928 16:02:19.625015 12027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:19.619703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:57631 (local address 127.0.0.1:46471)
0928 16:02:19.620457 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:19.620468 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:19.620505 (+ 37us) server_negotiation.cc:407] Connection header received
0928 16:02:19.621964 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:19.621984 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:19.622301 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:19.622597 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:19.623569 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:19.623575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:19.623587 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:19.623611 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:19.623662 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:19.623684 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:19.623843 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:19.623974 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 16:02:19.624210 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":357,"threads_started":1}
I20260928 16:02:19.625741 12024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78: Bootstrap starting.
I20260928 16:02:19.628513 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:19.628914 12024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:19.629853 12024 log.cc:824] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:19.632382 12024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78: No bootstrap required, opened a new log
I20260928 16:02:19.635094 12027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:19.632755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:58377 (local address 127.0.0.1:46471)
0928 16:02:19.632935 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:19.632940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:19.633279 (+ 339us) server_negotiation.cc:407] Connection header received
0928 16:02:19.633388 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:19.633392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:19.633447 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:19.633550 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:19.634412 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:19.634422 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:19.634427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:19.634445 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:19.634488 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:19.634508 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:19.634625 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:19.634730 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 16:02:19.634784 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:02:19.640753 12024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260928 16:02:19.641098 12024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:19.641167 12024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09ac9f7510cc4be9a39aaa51f5d00b78, State: Initialized, Role: FOLLOWER
I20260928 16:02:19.641727 12024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260928 16:02:19.641937 12024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:19.642030 12024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:19.642144 12024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:19.644137 12024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260928 16:02:19.644558 12024 leader_election.cc:302] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09ac9f7510cc4be9a39aaa51f5d00b78; no voters:
I20260928 16:02:19.645031 12024 leader_election.cc:288] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:19.645169 12034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:19.645830 12034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [term 1 LEADER]: Becoming Leader. State: Replica: 09ac9f7510cc4be9a39aaa51f5d00b78, State: Running, Role: LEADER
I20260928 16:02:19.646311 12034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260928 16:02:19.647019 12024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:19.648985 12036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09ac9f7510cc4be9a39aaa51f5d00b78. Latest consensus state: current_term: 1 leader_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } } }
I20260928 16:02:19.649427 12036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:19.650893 12035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ac9f7510cc4be9a39aaa51f5d00b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } } }
I20260928 16:02:19.651153 12035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:19.651245 12041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:19.656836 12041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:19.664790 12041 catalog_manager.cc:1347] Generated new cluster ID: 25d3270a4b7641f788b85782b64cc343
I20260928 16:02:19.664901 12041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:19.676452 12041 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:19.676884 12041 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:19.689390 12041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09ac9f7510cc4be9a39aaa51f5d00b78: Generated new TSK 0
I20260928 16:02:19.690311 12041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:19.697218 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11957
W20260928 16:02:19.698552 12019 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:57631
W20260928 16:02:19.699249 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 11957: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:19.699411 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 11957
2026-09-28T16:02:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1845 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:19 dist-test-slave-9cqd krb5kdc[12055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:19 dist-test-slave-9cqd krb5kdc[12055](info): set up 2 sockets
Sep 28 16:02:19 dist-test-slave-9cqd krb5kdc[12055](info): commencing operation
krb5kdc: starting...
W20260928 16:02:20.874212 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.007s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:21 dist-test-slave-9cqd krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:21.114941 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:21.434638 12068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:21.434942 12068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:21.435070 12068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:21.442117 12068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:21.442219 12068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:21.442269 12068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:21.442358 12068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:21.442448 12068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:21.451977 12068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36837
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:21.453290 12068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:21.454584 12068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:21.466521 12073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:21.466499 12076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:21.466457 12074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:21.467350 12068 server_base.cc:1034] running on GCE node
I20260928 16:02:21.468159 12068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:21.469523 12068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:21.470757 12068 hybrid_clock.cc:648] HybridClock initialized: now 1790611341470716 us; error 54 us; skew 500 ppm
I20260928 16:02:21.471118 12068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:21 dist-test-slave-9cqd krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611341, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:21.474272 12068 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:21.475734 12068 webserver.cc:460] Webserver started at http://127.0.0.1:39299/ using document root <none> and password file <none>
I20260928 16:02:21.476475 12068 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:21.476624 12068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:21.476933 12068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:21.479843 12068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "db750ebd62cc46acbc802dc4c713bb33"
format_stamp: "Formatted at 2026-09-28 16:02:21 on dist-test-slave-9cqd"
server_key: "a77ff33847a9f6280cb8f174bde58748"
server_key_iv: "048e6fd364d70308cac3ffdfcd92f1ac"
server_key_version: "encryptionkey@0"
I20260928 16:02:21.480679 12068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "db750ebd62cc46acbc802dc4c713bb33"
format_stamp: "Formatted at 2026-09-28 16:02:21 on dist-test-slave-9cqd"
server_key: "a77ff33847a9f6280cb8f174bde58748"
server_key_iv: "048e6fd364d70308cac3ffdfcd92f1ac"
server_key_version: "encryptionkey@0"
I20260928 16:02:21.485668 12068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:02:21.489078 12083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:21.490353 12068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 16:02:21.490538 12068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "db750ebd62cc46acbc802dc4c713bb33"
format_stamp: "Formatted at 2026-09-28 16:02:21 on dist-test-slave-9cqd"
server_key: "a77ff33847a9f6280cb8f174bde58748"
server_key_iv: "048e6fd364d70308cac3ffdfcd92f1ac"
server_key_version: "encryptionkey@0"
I20260928 16:02:21.490731 12068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:21.515208 12068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:21.519068 12068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:21.519345 12068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:21.537865 12068 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36837
I20260928 16:02:21.537914 12134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36837 every 8 connection(s)
I20260928 16:02:21.539175 12068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:21.542768 12135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:21.547891 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12068
I20260928 16:02:21.548094 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:21.548524 23031 external_mini_cluster.cc:1428] Setting key 8d55d9126d83dc022692db5e97cfad62
I20260928 16:02:21.554823 12135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33: Bootstrap starting.
I20260928 16:02:21.558503 12135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:21.559559 12135 log.cc:824] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:21.562403 12135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33: No bootstrap required, opened a new log
Sep 28 16:02:21 dist-test-slave-9cqd krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 16:02:21.572944 12135 raft_consensus.cc:348] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db750ebd62cc46acbc802dc4c713bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36837 } }
I20260928 16:02:21.573400 12135 raft_consensus.cc:374] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:21.573510 12135 raft_consensus.cc:729] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db750ebd62cc46acbc802dc4c713bb33, State: Initialized, Role: FOLLOWER
I20260928 16:02:21.573662 12138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:21.550316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:35485 (local address 127.0.0.1:36837)
0928 16:02:21.551205 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:21.551220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:21.551276 (+ 56us) server_negotiation.cc:407] Connection header received
0928 16:02:21.553691 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:21.553716 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:21.554199 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:21.554665 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:21.556076 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:21.558133 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:21.559454 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:21.560518 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:21.560647 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:21.565025 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:21.565035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:21.565053 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:21.565093 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:21.569126 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:21.569912 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:21.569921 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:21.569929 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:21.570122 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:21.570531 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:21.570536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:21.570539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:21.571261 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:21.571578 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:21.571929 (+ 351us) server_negotiation.cc:299] Negotiation successful
0928 16:02:21.572302 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":371,"threads_started":1}
I20260928 16:02:21.574209 12135 consensus_queue.cc:260] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db750ebd62cc46acbc802dc4c713bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36837 } }
I20260928 16:02:21.574508 12135 raft_consensus.cc:388] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:21.574652 12135 raft_consensus.cc:482] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:21.574839 12135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:21.577508 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:21.577581 12135 raft_consensus.cc:504] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db750ebd62cc46acbc802dc4c713bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36837 } }
I20260928 16:02:21.578099 12135 leader_election.cc:302] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db750ebd62cc46acbc802dc4c713bb33; no voters:
I20260928 16:02:21.578745 12135 leader_election.cc:288] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:21.578847 12140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:21.581494 12135 sys_catalog.cc:564] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:21.584412 12140 raft_consensus.cc:686] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [term 1 LEADER]: Becoming Leader. State: Replica: db750ebd62cc46acbc802dc4c713bb33, State: Running, Role: LEADER
I20260928 16:02:21.585220 12140 consensus_queue.cc:237] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db750ebd62cc46acbc802dc4c713bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36837 } }
I20260928 16:02:21.589309 12141 sys_catalog.cc:455] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db750ebd62cc46acbc802dc4c713bb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db750ebd62cc46acbc802dc4c713bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36837 } } }
I20260928 16:02:21.589785 12141 sys_catalog.cc:458] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:21.591809 12142 sys_catalog.cc:455] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db750ebd62cc46acbc802dc4c713bb33. Latest consensus state: current_term: 1 leader_uuid: "db750ebd62cc46acbc802dc4c713bb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db750ebd62cc46acbc802dc4c713bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36837 } } }
I20260928 16:02:21.592283 12142 sys_catalog.cc:458] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:21.593683 12153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:21.600247 12153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:21.605520 12138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:21.586210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:38365 (local address 127.0.0.1:36837)
0928 16:02:21.586432 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:21.586441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:21.587463 (+ 1022us) server_negotiation.cc:407] Connection header received
0928 16:02:21.587699 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:21.587707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:21.587813 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:21.587931 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:21.590247 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:21.591553 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:21.593706 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:21.594309 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:21.594505 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:21.597827 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:21.597833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:21.597838 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:21.597875 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:21.601748 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:21.602788 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:21.602794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:21.602797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:21.602933 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:21.603524 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:21.603529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:21.603533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:21.603969 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:21.604279 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:21.604994 (+ 715us) server_negotiation.cc:299] Negotiation successful
0928 16:02:21.605177 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 16:02:21.610581 12153 catalog_manager.cc:1347] Generated new cluster ID: 4eaf2217869043a2a798857339c07c2f
I20260928 16:02:21.610678 12153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:21.626798 12153 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:21.628532 12153 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:21.645211 12153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db750ebd62cc46acbc802dc4c713bb33: Generated new TSK 0
I20260928 16:02:21.646157 12153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:21.661381 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12068
I20260928 16:02:21.662463 12130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:21.755429 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12068
W20260928 16:02:21.772480 12053 connection.cc:535] client connection to 127.0.0.1:36837 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36837): Connection reset by peer (error 104)
2026-09-28T16:02:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2059 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:21 dist-test-slave-9cqd krb5kdc[12167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:21 dist-test-slave-9cqd krb5kdc[12167](info): set up 2 sockets
Sep 28 16:02:21 dist-test-slave-9cqd krb5kdc[12167](info): commencing operation
krb5kdc: starting...
W20260928 16:02:22.858321 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:23 dist-test-slave-9cqd krb5kdc[12167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:23.091382 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32909
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:23.411198 12180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:23.411478 12180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:23.411667 12180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:23.418679 12180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:23.418776 12180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:23.418895 12180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:23.419024 12180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:23.419101 12180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:23.428699 12180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32909
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:23.429924 12180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:23.431269 12180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:23.442862 12186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:23.442889 12188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:23.442950 12185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:23.443885 12180 server_base.cc:1034] running on GCE node
I20260928 16:02:23.444751 12180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:23.446096 12180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:23.447328 12180 hybrid_clock.cc:648] HybridClock initialized: now 1790611343447288 us; error 54 us; skew 500 ppm
I20260928 16:02:23.447679 12180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:23 dist-test-slave-9cqd krb5kdc[12167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611343, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:23.450781 12180 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:23.452237 12180 webserver.cc:460] Webserver started at http://127.0.0.1:45115/ using document root <none> and password file <none>
I20260928 16:02:23.452912 12180 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:23.453061 12180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:23.453369 12180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:23.456328 12180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "cfe318785284429eb27af855e90ca480"
format_stamp: "Formatted at 2026-09-28 16:02:23 on dist-test-slave-9cqd"
server_key: "252d418e09c9061a633f43173c3926ad"
server_key_iv: "caf9cdb19471bcb35ba37c099ba85ef3"
server_key_version: "encryptionkey@0"
I20260928 16:02:23.457087 12180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "cfe318785284429eb27af855e90ca480"
format_stamp: "Formatted at 2026-09-28 16:02:23 on dist-test-slave-9cqd"
server_key: "252d418e09c9061a633f43173c3926ad"
server_key_iv: "caf9cdb19471bcb35ba37c099ba85ef3"
server_key_version: "encryptionkey@0"
I20260928 16:02:23.462073 12180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:02:23.465371 12195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:23.466562 12180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:02:23.466738 12180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "cfe318785284429eb27af855e90ca480"
format_stamp: "Formatted at 2026-09-28 16:02:23 on dist-test-slave-9cqd"
server_key: "252d418e09c9061a633f43173c3926ad"
server_key_iv: "caf9cdb19471bcb35ba37c099ba85ef3"
server_key_version: "encryptionkey@0"
I20260928 16:02:23.466938 12180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:23.482378 12180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:23.486930 12180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:23.487226 12180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:23.505875 12180 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32909
I20260928 16:02:23.505906 12246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32909 every 8 connection(s)
I20260928 16:02:23.507292 12180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:23.511019 12247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:23.512748 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12180
I20260928 16:02:23.512951 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:23.513484 23031 external_mini_cluster.cc:1428] Setting key 0f076ba423e32c304915693d16130c87
I20260928 16:02:23.524837 12247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480: Bootstrap starting.
I20260928 16:02:23.527576 12247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:23.528574 12247 log.cc:824] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:23.531437 12247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480: No bootstrap required, opened a new log
I20260928 16:02:23.531957 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:23.515254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:51997 (local address 127.0.0.1:32909)
0928 16:02:23.516076 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:23.516095 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:23.516149 (+ 54us) server_negotiation.cc:407] Connection header received
0928 16:02:23.517688 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:23.517708 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:23.518098 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:23.518420 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:23.519051 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:23.520352 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:23.522292 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:23.522970 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:23.523092 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:23.529689 (+ 6597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:23.529700 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:23.529719 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:23.530265 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:23.530338 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:23.530362 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:23.530489 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:23.530780 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 16:02:23.530966 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":369,"threads_started":1}
I20260928 16:02:23.535789 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:23.540858 12247 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe318785284429eb27af855e90ca480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32909 } }
I20260928 16:02:23.541363 12247 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:23.541486 12247 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe318785284429eb27af855e90ca480, State: Initialized, Role: FOLLOWER
I20260928 16:02:23.542131 12247 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe318785284429eb27af855e90ca480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32909 } }
I20260928 16:02:23.542411 12247 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:23.542570 12247 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:23.542763 12247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:23.545536 12247 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe318785284429eb27af855e90ca480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32909 } }
I20260928 16:02:23.546175 12247 leader_election.cc:302] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfe318785284429eb27af855e90ca480; no voters:
I20260928 16:02:23.546870 12247 leader_election.cc:288] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:23.546993 12257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:23.548244 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:23.541014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:43955 (local address 127.0.0.1:32909)
0928 16:02:23.541206 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:23.541215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:23.541625 (+ 410us) server_negotiation.cc:407] Connection header received
0928 16:02:23.541890 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:23.541895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:23.541972 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:23.542075 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:23.542673 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:23.543886 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:23.545184 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:23.545888 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:23.546020 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:23.547227 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:23.547237 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:23.547245 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:23.547282 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:23.547353 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:23.547380 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:23.547524 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:23.547862 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 16:02:23.547915 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:02:23.549237 12247 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:23.553833 12257 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [term 1 LEADER]: Becoming Leader. State: Replica: cfe318785284429eb27af855e90ca480, State: Running, Role: LEADER
I20260928 16:02:23.554369 12257 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe318785284429eb27af855e90ca480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32909 } }
I20260928 16:02:23.557209 12259 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfe318785284429eb27af855e90ca480. Latest consensus state: current_term: 1 leader_uuid: "cfe318785284429eb27af855e90ca480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe318785284429eb27af855e90ca480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32909 } } }
I20260928 16:02:23.557302 12258 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfe318785284429eb27af855e90ca480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe318785284429eb27af855e90ca480" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32909 } } }
I20260928 16:02:23.557559 12259 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:23.557637 12258 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:23.558892 12272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:23.563828 12272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:23.571836 12272 catalog_manager.cc:1347] Generated new cluster ID: 5ed29e1fe97f45b99788c490fa29ed0e
I20260928 16:02:23.571950 12272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:23.590516 12272 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:23.591773 12272 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:23.599054 12272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfe318785284429eb27af855e90ca480: Generated new TSK 0
I20260928 16:02:23.599812 12272 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:23.611755 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12180
W20260928 16:02:23.612882 12242 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:51997
W20260928 16:02:23.613686 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12180: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:23.613818 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12180
W20260928 16:02:23.630662 12165 connection.cc:535] client connection to 127.0.0.1:32909 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32909): Connection reset by peer (error 104)
2026-09-28T16:02:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1858 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:23 dist-test-slave-9cqd krb5kdc[12279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:23 dist-test-slave-9cqd krb5kdc[12279](info): set up 2 sockets
Sep 28 16:02:23 dist-test-slave-9cqd krb5kdc[12279](info): commencing operation
krb5kdc: starting...
W20260928 16:02:24.726393 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:24 dist-test-slave-9cqd krb5kdc[12279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 16:02:24.967283 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:25.286445 12292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:25.286795 12292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:25.286965 12292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:25.294050 12292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:25.294205 12292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:25.294315 12292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:25.294426 12292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:25.294513 12292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:25.304075 12292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:25.305379 12292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:25.306684 12292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:25.318421 12297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:25.318457 12300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:25.318373 12298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:25.319376 12292 server_base.cc:1034] running on GCE node
I20260928 16:02:25.320197 12292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:25.321499 12292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:25.322733 12292 hybrid_clock.cc:648] HybridClock initialized: now 1790611345322701 us; error 45 us; skew 500 ppm
I20260928 16:02:25.323060 12292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:25 dist-test-slave-9cqd krb5kdc[12279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611345, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:25.333416 12292 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 16:02:25.334801 12292 webserver.cc:460] Webserver started at http://127.0.0.1:41587/ using document root <none> and password file <none>
I20260928 16:02:25.335457 12292 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:25.335608 12292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:25.335917 12292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:25.338852 12292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "41b700c32d2d49c2802cc61d50eacb58"
format_stamp: "Formatted at 2026-09-28 16:02:25 on dist-test-slave-9cqd"
server_key: "68cc678df130379bef713caf27a64d65"
server_key_iv: "6dc1a91a8af2735a7170a8f05989135c"
server_key_version: "encryptionkey@0"
I20260928 16:02:25.339609 12292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "41b700c32d2d49c2802cc61d50eacb58"
format_stamp: "Formatted at 2026-09-28 16:02:25 on dist-test-slave-9cqd"
server_key: "68cc678df130379bef713caf27a64d65"
server_key_iv: "6dc1a91a8af2735a7170a8f05989135c"
server_key_version: "encryptionkey@0"
I20260928 16:02:25.344533 12292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 16:02:25.347720 12307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:25.349017 12292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:25.349201 12292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "41b700c32d2d49c2802cc61d50eacb58"
format_stamp: "Formatted at 2026-09-28 16:02:25 on dist-test-slave-9cqd"
server_key: "68cc678df130379bef713caf27a64d65"
server_key_iv: "6dc1a91a8af2735a7170a8f05989135c"
server_key_version: "encryptionkey@0"
I20260928 16:02:25.349401 12292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:25.354838 12292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:25.358537 12292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:25.358836 12292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:25.377053 12292 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45419
I20260928 16:02:25.377133 12358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45419 every 8 connection(s)
I20260928 16:02:25.378410 12292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:25.379933 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12292
I20260928 16:02:25.380153 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:25.380695 23031 external_mini_cluster.cc:1428] Setting key 42e64da7db1a1db1c55b16850d8c674f
I20260928 16:02:25.382866 12359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:25.388331 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:25.382497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:45901 (local address 127.0.0.1:45419)
0928 16:02:25.383269 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:25.383280 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:25.383325 (+ 45us) server_negotiation.cc:407] Connection header received
0928 16:02:25.384958 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:25.384978 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:25.385309 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:25.385687 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:25.386696 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:25.386701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:25.386714 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:25.386731 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:25.386803 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:25.386821 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:25.387077 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:25.387219 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 16:02:25.387406 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":314,"threads_started":1}
I20260928 16:02:25.391716 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:25.394048 12359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58: Bootstrap starting.
I20260928 16:02:25.397538 12359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:25.398439 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:25.395910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:51257 (local address 127.0.0.1:45419)
0928 16:02:25.396120 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:25.396129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:25.396587 (+ 458us) server_negotiation.cc:407] Connection header received
0928 16:02:25.396744 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:25.396747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:25.396791 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:25.396872 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:25.397786 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:25.397792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:25.397796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:25.397816 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:25.397865 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:25.397894 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:25.398031 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:25.398114 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 16:02:25.398170 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:02:25.398852 12359 log.cc:824] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:25.401513 12359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58: No bootstrap required, opened a new log
I20260928 16:02:25.408874 12359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b700c32d2d49c2802cc61d50eacb58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260928 16:02:25.409230 12359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:25.409339 12359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41b700c32d2d49c2802cc61d50eacb58, State: Initialized, Role: FOLLOWER
I20260928 16:02:25.409875 12359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b700c32d2d49c2802cc61d50eacb58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260928 16:02:25.410115 12359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:25.410254 12359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:25.410434 12359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:25.412596 12359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b700c32d2d49c2802cc61d50eacb58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260928 16:02:25.413070 12359 leader_election.cc:302] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41b700c32d2d49c2802cc61d50eacb58; no voters:
I20260928 16:02:25.413615 12359 leader_election.cc:288] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:25.413743 12370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:25.414978 12370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [term 1 LEADER]: Becoming Leader. State: Replica: 41b700c32d2d49c2802cc61d50eacb58, State: Running, Role: LEADER
I20260928 16:02:25.415499 12370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b700c32d2d49c2802cc61d50eacb58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260928 16:02:25.415729 12359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:25.418577 12372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41b700c32d2d49c2802cc61d50eacb58. Latest consensus state: current_term: 1 leader_uuid: "41b700c32d2d49c2802cc61d50eacb58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b700c32d2d49c2802cc61d50eacb58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } } }
I20260928 16:02:25.418982 12372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:25.419448 12371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41b700c32d2d49c2802cc61d50eacb58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b700c32d2d49c2802cc61d50eacb58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } } }
I20260928 16:02:25.419726 12371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:25.423372 12379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:25.428520 12379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:25.436794 12379 catalog_manager.cc:1347] Generated new cluster ID: aba2b0977ca14185a5cad280ef9d37e2
I20260928 16:02:25.436898 12379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:25.461030 12379 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:25.461493 12379 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:25.480476 12379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41b700c32d2d49c2802cc61d50eacb58: Generated new TSK 0
I20260928 16:02:25.481529 12379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:25.486348 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12292
W20260928 16:02:25.487782 12354 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:45901
W20260928 16:02:25.488473 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12292: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:25.488624 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12292
2026-09-28T16:02:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1874 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:25 dist-test-slave-9cqd krb5kdc[12391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:25 dist-test-slave-9cqd krb5kdc[12391](info): set up 2 sockets
Sep 28 16:02:25 dist-test-slave-9cqd krb5kdc[12391](info): commencing operation
krb5kdc: starting...
W20260928 16:02:26.586274 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:26 dist-test-slave-9cqd krb5kdc[12391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:26.813227 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:33733
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:33733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:27.131461 12404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:27.131732 12404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:27.131824 12404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:27.138880 12404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:27.138976 12404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:27.139021 12404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:27.139081 12404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:27.139123 12404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:27.148521 12404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:33733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:33733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:27.149719 12404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:27.150956 12404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:27.162614 12409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:27.162647 12412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:27.162700 12410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:27.168642 12404 server_base.cc:1034] running on GCE node
I20260928 16:02:27.169338 12404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:27.170593 12404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:27.171790 12404 hybrid_clock.cc:648] HybridClock initialized: now 1790611347171752 us; error 36 us; skew 500 ppm
I20260928 16:02:27.172181 12404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:27 dist-test-slave-9cqd krb5kdc[12391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611347, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:27.175444 12404 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:27.176968 12404 webserver.cc:460] Webserver started at http://127.22.125.254:38045/ using document root <none> and password file <none>
I20260928 16:02:27.177654 12404 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:27.177800 12404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:27.178086 12404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:27.180992 12404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "e78c452780844cd0a0e608009a8a1381"
format_stamp: "Formatted at 2026-09-28 16:02:27 on dist-test-slave-9cqd"
server_key: "88b418f761d0e174490b72ed636bf0eb"
server_key_iv: "8606d1e6d054c343142424cb9877b98e"
server_key_version: "encryptionkey@0"
I20260928 16:02:27.181742 12404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "e78c452780844cd0a0e608009a8a1381"
format_stamp: "Formatted at 2026-09-28 16:02:27 on dist-test-slave-9cqd"
server_key: "88b418f761d0e174490b72ed636bf0eb"
server_key_iv: "8606d1e6d054c343142424cb9877b98e"
server_key_version: "encryptionkey@0"
I20260928 16:02:27.186785 12404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 16:02:27.190102 12419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:27.191345 12404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 16:02:27.191526 12404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "e78c452780844cd0a0e608009a8a1381"
format_stamp: "Formatted at 2026-09-28 16:02:27 on dist-test-slave-9cqd"
server_key: "88b418f761d0e174490b72ed636bf0eb"
server_key_iv: "8606d1e6d054c343142424cb9877b98e"
server_key_version: "encryptionkey@0"
I20260928 16:02:27.191716 12404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:27.224884 12404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:27.229230 12404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:27.229524 12404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:27.247857 12404 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:33733
I20260928 16:02:27.247879 12470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:33733 every 8 connection(s)
I20260928 16:02:27.249223 12404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:27.252887 12471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:27.255364 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12404
I20260928 16:02:27.255566 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:27.256146 23031 external_mini_cluster.cc:1428] Setting key a29e32dd4bfacb5e632158c74941dac1
I20260928 16:02:27.265026 12471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381: Bootstrap starting.
I20260928 16:02:27.268740 12471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:27.270043 12471 log.cc:824] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381: Log is configured to *not* fsync() on all Append() calls
Sep 28 16:02:27 dist-test-slave-9cqd krb5kdc[12391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:02:27.272876 12471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381: No bootstrap required, opened a new log
I20260928 16:02:27.279171 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:27.257904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49025 (local address 127.22.125.254:33733)
0928 16:02:27.258718 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:27.258729 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:27.258769 (+ 40us) server_negotiation.cc:407] Connection header received
0928 16:02:27.260455 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:27.260476 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:27.260832 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:27.261155 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:27.262892 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:27.264214 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:27.266070 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:27.266716 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:27.266841 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:27.271828 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:27.271835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:27.271848 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:27.271881 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:27.275045 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:27.275825 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:27.275831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:27.275836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:27.276010 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:27.276508 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:27.276517 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:27.276519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:27.277067 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:27.277409 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:27.277909 (+ 500us) server_negotiation.cc:299] Negotiation successful
0928 16:02:27.278231 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":409,"threads_started":1}
I20260928 16:02:27.280613 12471 raft_consensus.cc:348] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78c452780844cd0a0e608009a8a1381" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33733 } }
I20260928 16:02:27.280984 12471 raft_consensus.cc:374] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:27.281108 12471 raft_consensus.cc:729] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78c452780844cd0a0e608009a8a1381, State: Initialized, Role: FOLLOWER
I20260928 16:02:27.281674 12471 consensus_queue.cc:260] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78c452780844cd0a0e608009a8a1381" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33733 } }
I20260928 16:02:27.281940 12471 raft_consensus.cc:388] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:27.282109 12471 raft_consensus.cc:482] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:27.282311 12471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:27.282866 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:27.284615 12471 raft_consensus.cc:504] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78c452780844cd0a0e608009a8a1381" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33733 } }
I20260928 16:02:27.285108 12471 leader_election.cc:302] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e78c452780844cd0a0e608009a8a1381; no voters:
I20260928 16:02:27.285799 12471 leader_election.cc:288] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:27.285888 12478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:27.288431 12471 sys_catalog.cc:564] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:27.292860 12478 raft_consensus.cc:686] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [term 1 LEADER]: Becoming Leader. State: Replica: e78c452780844cd0a0e608009a8a1381, State: Running, Role: LEADER
I20260928 16:02:27.293612 12478 consensus_queue.cc:237] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78c452780844cd0a0e608009a8a1381" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33733 } }
I20260928 16:02:27.297021 12481 sys_catalog.cc:455] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e78c452780844cd0a0e608009a8a1381. Latest consensus state: current_term: 1 leader_uuid: "e78c452780844cd0a0e608009a8a1381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78c452780844cd0a0e608009a8a1381" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33733 } } }
I20260928 16:02:27.297487 12481 sys_catalog.cc:458] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:27.298069 12482 sys_catalog.cc:455] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e78c452780844cd0a0e608009a8a1381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78c452780844cd0a0e608009a8a1381" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 33733 } } }
I20260928 16:02:27.298389 12482 sys_catalog.cc:458] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:27.300628 12488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:27.307296 12488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:27.312479 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:27.293372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44649 (local address 127.22.125.254:33733)
0928 16:02:27.293533 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:27.293559 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:27.294122 (+ 563us) server_negotiation.cc:407] Connection header received
0928 16:02:27.294311 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:27.294317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:27.294409 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:27.294488 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:27.296504 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:27.297686 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:27.300002 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:27.301077 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:27.301215 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:27.304339 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:27.304348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:27.304354 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:27.304399 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:27.308307 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:27.309249 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:27.309254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:27.309258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:27.309394 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:27.309947 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:27.309953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:27.309956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:27.310343 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:27.310603 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:27.311796 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0928 16:02:27.311975 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 16:02:27.320122 12488 catalog_manager.cc:1347] Generated new cluster ID: 400870a82ef84338b535c66a100297c8
I20260928 16:02:27.320487 12488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:27.339656 12488 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:27.340950 12488 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:27.346549 12488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e78c452780844cd0a0e608009a8a1381: Generated new TSK 0
I20260928 16:02:27.347308 12488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:27.357618 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12404
I20260928 16:02:27.359232 12466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:27.455039 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12404
W20260928 16:02:27.472450 12389 connection.cc:535] client connection to 127.22.125.254:33733 recv error: Network error: failed to read from TLS socket (remote: 127.22.125.254:33733): Connection reset by peer (error 104)
2026-09-28T16:02:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1966 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:27 dist-test-slave-9cqd krb5kdc[12505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:27 dist-test-slave-9cqd krb5kdc[12505](info): set up 2 sockets
Sep 28 16:02:27 dist-test-slave-9cqd krb5kdc[12505](info): commencing operation
krb5kdc: starting...
W20260928 16:02:28.554289 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:28 dist-test-slave-9cqd krb5kdc[12505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:28.793315 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:46451
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:42305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:46451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:29.111150 12518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:29.111502 12518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:29.111678 12518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:29.118713 12518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:29.118860 12518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:29.118966 12518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:29.119078 12518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:29.119163 12518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:29.128710 12518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:42305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:46451
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:46451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:29.129940 12518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:29.131243 12518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:29.142803 12524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:29.142886 12526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:29.143301 12523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:29.143899 12518 server_base.cc:1034] running on GCE node
I20260928 16:02:29.144718 12518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:29.146080 12518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:29.147320 12518 hybrid_clock.cc:648] HybridClock initialized: now 1790611349147273 us; error 53 us; skew 500 ppm
I20260928 16:02:29.147667 12518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:29 dist-test-slave-9cqd krb5kdc[12505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611349, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:29.150811 12518 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:29.152256 12518 webserver.cc:460] Webserver started at http://127.22.125.254:35759/ using document root <none> and password file <none>
I20260928 16:02:29.152926 12518 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:29.153077 12518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:29.153404 12518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:29.156466 12518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "e190b0ea4312486b81de20bbfaef3447"
format_stamp: "Formatted at 2026-09-28 16:02:29 on dist-test-slave-9cqd"
server_key: "427c9933682b8c99ddf6f80c9ee8cc28"
server_key_iv: "dd3a3ac9501a363393e7ae2798089a47"
server_key_version: "encryptionkey@0"
I20260928 16:02:29.157209 12518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "e190b0ea4312486b81de20bbfaef3447"
format_stamp: "Formatted at 2026-09-28 16:02:29 on dist-test-slave-9cqd"
server_key: "427c9933682b8c99ddf6f80c9ee8cc28"
server_key_iv: "dd3a3ac9501a363393e7ae2798089a47"
server_key_version: "encryptionkey@0"
I20260928 16:02:29.162173 12518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 16:02:29.165518 12533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:29.166765 12518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 16:02:29.166944 12518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "e190b0ea4312486b81de20bbfaef3447"
format_stamp: "Formatted at 2026-09-28 16:02:29 on dist-test-slave-9cqd"
server_key: "427c9933682b8c99ddf6f80c9ee8cc28"
server_key_iv: "dd3a3ac9501a363393e7ae2798089a47"
server_key_version: "encryptionkey@0"
I20260928 16:02:29.167132 12518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:29.180588 12518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:29.184324 12518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:29.184593 12518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:29.202911 12518 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:46451
I20260928 16:02:29.202935 12584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:46451 every 8 connection(s)
I20260928 16:02:29.204380 12518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:29.206343 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12518
I20260928 16:02:29.206543 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:29.207062 23031 external_mini_cluster.cc:1428] Setting key 6856b3194201a6b3f7dcd226b4c2e602
I20260928 16:02:29.208501 12585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:29.221858 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:29.208890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34511 (local address 127.22.125.254:46451)
0928 16:02:29.209693 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:29.209707 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:29.209756 (+ 49us) server_negotiation.cc:407] Connection header received
0928 16:02:29.211663 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:29.211690 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:29.212225 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:29.212668 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:29.213192 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:29.215231 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:29.216716 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:29.217736 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:29.217937 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:29.219066 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:29.219081 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:29.219098 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:29.219872 (+ 774us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:29.219966 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:29.219997 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:29.220190 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:29.220403 (+ 213us) server_negotiation.cc:299] Negotiation successful
0928 16:02:29.220585 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":339,"threads_started":1}
I20260928 16:02:29.223243 12585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447: Bootstrap starting.
I20260928 16:02:29.225596 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:29.226680 12585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:29.227908 12585 log.cc:824] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:29.231576 12585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447: No bootstrap required, opened a new log
I20260928 16:02:29.240309 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:29.233499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50841 (local address 127.22.125.254:46451)
0928 16:02:29.233718 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:29.233731 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:29.234199 (+ 468us) server_negotiation.cc:407] Connection header received
0928 16:02:29.234384 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:29.234387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:29.234450 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:29.234533 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:29.235108 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:29.235885 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:29.237772 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:29.238232 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:29.238329 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:29.239487 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:29.239492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:29.239496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:29.239514 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:29.239553 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:29.239567 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:29.239656 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:29.239864 (+ 208us) server_negotiation.cc:299] Negotiation successful
0928 16:02:29.239930 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:02:29.241808 12585 raft_consensus.cc:348] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190b0ea4312486b81de20bbfaef3447" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 46451 } }
I20260928 16:02:29.242187 12585 raft_consensus.cc:374] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:29.242290 12585 raft_consensus.cc:729] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e190b0ea4312486b81de20bbfaef3447, State: Initialized, Role: FOLLOWER
I20260928 16:02:29.242838 12585 consensus_queue.cc:260] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190b0ea4312486b81de20bbfaef3447" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 46451 } }
I20260928 16:02:29.243077 12585 raft_consensus.cc:388] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:29.243247 12585 raft_consensus.cc:482] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:29.243438 12585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:29.245599 12585 raft_consensus.cc:504] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190b0ea4312486b81de20bbfaef3447" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 46451 } }
I20260928 16:02:29.246165 12585 leader_election.cc:302] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e190b0ea4312486b81de20bbfaef3447; no voters:
I20260928 16:02:29.246809 12585 leader_election.cc:288] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:29.246959 12595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:29.247663 12595 raft_consensus.cc:686] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [term 1 LEADER]: Becoming Leader. State: Replica: e190b0ea4312486b81de20bbfaef3447, State: Running, Role: LEADER
I20260928 16:02:29.248229 12595 consensus_queue.cc:237] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190b0ea4312486b81de20bbfaef3447" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 46451 } }
I20260928 16:02:29.248889 12585 sys_catalog.cc:564] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:29.251147 12597 sys_catalog.cc:455] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e190b0ea4312486b81de20bbfaef3447. Latest consensus state: current_term: 1 leader_uuid: "e190b0ea4312486b81de20bbfaef3447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190b0ea4312486b81de20bbfaef3447" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 46451 } } }
I20260928 16:02:29.251586 12597 sys_catalog.cc:458] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:29.253783 12596 sys_catalog.cc:455] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e190b0ea4312486b81de20bbfaef3447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190b0ea4312486b81de20bbfaef3447" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 46451 } } }
I20260928 16:02:29.254050 12596 sys_catalog.cc:458] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:29.254555 12602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:29.259219 12602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:29.267424 12602 catalog_manager.cc:1347] Generated new cluster ID: 58fa3893b4e34ae1abc0e40f4be0fecf
I20260928 16:02:29.267522 12602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:29.289496 12602 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:29.290915 12602 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:29.301999 12602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e190b0ea4312486b81de20bbfaef3447: Generated new TSK 0
I20260928 16:02:29.302778 12602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:29.315513 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12518
W20260928 16:02:29.317063 12580 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34511
W20260928 16:02:29.317791 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12518: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:29.317925 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12518
W20260928 16:02:29.334404 12503 connection.cc:535] client connection to 127.22.125.254:46451 recv error: Network error: failed to read from TLS socket (remote: 127.22.125.254:46451): Connection reset by peer (error 104)
2026-09-28T16:02:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1862 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:29 dist-test-slave-9cqd krb5kdc[12616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:29 dist-test-slave-9cqd krb5kdc[12616](info): set up 2 sockets
Sep 28 16:02:29 dist-test-slave-9cqd krb5kdc[12616](info): commencing operation
krb5kdc: starting...
W20260928 16:02:30.410539 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:30 dist-test-slave-9cqd krb5kdc[12616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:30.657603 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45025
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:40373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:45025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:30.982417 12630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:30.982743 12630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:30.982913 12630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:30.990124 12630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:30.990248 12630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:30.990352 12630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:30.990454 12630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:30.990542 12630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:31.000288 12630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:40373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:45025
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:45025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:31.001554 12630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:31.002873 12630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:31.014652 12635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:31.014680 12638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:31.014703 12636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:31.015770 12630 server_base.cc:1034] running on GCE node
I20260928 16:02:31.016568 12630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:31.017927 12630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:31.019126 12630 hybrid_clock.cc:648] HybridClock initialized: now 1790611351019093 us; error 32 us; skew 500 ppm
I20260928 16:02:31.019481 12630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:31 dist-test-slave-9cqd krb5kdc[12616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611351, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:31.028934 12630 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:31.030447 12630 webserver.cc:460] Webserver started at http://127.22.125.254:41099/ using document root <none> and password file <none>
I20260928 16:02:31.031123 12630 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:31.031296 12630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:31.031596 12630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:31.034612 12630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "a52f6256b4a44cf4b6b63a1affd29a00"
format_stamp: "Formatted at 2026-09-28 16:02:31 on dist-test-slave-9cqd"
server_key: "b7b2390d50319f984cc60ca4a29996e3"
server_key_iv: "17b2fbc58197c79900e60bc2a69a8afc"
server_key_version: "encryptionkey@0"
I20260928 16:02:31.035364 12630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "a52f6256b4a44cf4b6b63a1affd29a00"
format_stamp: "Formatted at 2026-09-28 16:02:31 on dist-test-slave-9cqd"
server_key: "b7b2390d50319f984cc60ca4a29996e3"
server_key_iv: "17b2fbc58197c79900e60bc2a69a8afc"
server_key_version: "encryptionkey@0"
I20260928 16:02:31.040442 12630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260928 16:02:31.043630 12645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:31.045008 12630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:31.045192 12630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "a52f6256b4a44cf4b6b63a1affd29a00"
format_stamp: "Formatted at 2026-09-28 16:02:31 on dist-test-slave-9cqd"
server_key: "b7b2390d50319f984cc60ca4a29996e3"
server_key_iv: "17b2fbc58197c79900e60bc2a69a8afc"
server_key_version: "encryptionkey@0"
I20260928 16:02:31.045392 12630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:31.051136 12630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:31.054991 12630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:31.055290 12630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:31.073524 12630 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:45025
I20260928 16:02:31.073554 12696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:45025 every 8 connection(s)
I20260928 16:02:31.074885 12630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:31.078514 12697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:31.080822 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12630
I20260928 16:02:31.080998 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:31.081530 23031 external_mini_cluster.cc:1428] Setting key 9d9813277a1bb5b266ec268e88b3bcc9
I20260928 16:02:31.089306 12700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:31.083303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48177 (local address 127.22.125.254:45025)
0928 16:02:31.084161 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:31.084176 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:31.084222 (+ 46us) server_negotiation.cc:407] Connection header received
0928 16:02:31.085879 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:31.085902 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:31.086235 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:31.086546 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:31.087653 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:31.087662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:31.087674 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:31.087692 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:31.087751 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:31.087774 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:31.087950 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:31.088153 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 16:02:31.088378 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":407,"threads_started":1}
I20260928 16:02:31.091177 12697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00: Bootstrap starting.
I20260928 16:02:31.093017 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:31.094532 12697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:31.095541 12697 log.cc:824] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:31.098462 12697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00: No bootstrap required, opened a new log
I20260928 16:02:31.099963 12700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:31.097462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48419 (local address 127.22.125.254:45025)
0928 16:02:31.097640 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:31.097647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:31.098141 (+ 494us) server_negotiation.cc:407] Connection header received
0928 16:02:31.098266 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:31.098270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:31.098332 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:31.098443 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:31.099339 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:31.099346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:31.099350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:31.099368 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:31.099411 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:31.099430 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:31.099559 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:31.099627 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 16:02:31.099682 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:02:31.106362 12697 raft_consensus.cc:348] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45025 } }
I20260928 16:02:31.106688 12697 raft_consensus.cc:374] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:31.106760 12697 raft_consensus.cc:729] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52f6256b4a44cf4b6b63a1affd29a00, State: Initialized, Role: FOLLOWER
I20260928 16:02:31.107232 12697 consensus_queue.cc:260] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45025 } }
I20260928 16:02:31.107429 12697 raft_consensus.cc:388] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:31.107570 12697 raft_consensus.cc:482] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:31.107695 12697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:31.109846 12697 raft_consensus.cc:504] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45025 } }
I20260928 16:02:31.110272 12697 leader_election.cc:302] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a52f6256b4a44cf4b6b63a1affd29a00; no voters:
I20260928 16:02:31.110785 12697 leader_election.cc:288] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:31.110949 12707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:31.111594 12707 raft_consensus.cc:686] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [term 1 LEADER]: Becoming Leader. State: Replica: a52f6256b4a44cf4b6b63a1affd29a00, State: Running, Role: LEADER
I20260928 16:02:31.112223 12707 consensus_queue.cc:237] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45025 } }
I20260928 16:02:31.112994 12697 sys_catalog.cc:564] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:31.115231 12709 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a52f6256b4a44cf4b6b63a1affd29a00. Latest consensus state: current_term: 1 leader_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45025 } } }
I20260928 16:02:31.115711 12709 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:31.115718 12708 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52f6256b4a44cf4b6b63a1affd29a00" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 45025 } } }
I20260928 16:02:31.116245 12708 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:31.117470 12716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:31.122660 12716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:31.131194 12716 catalog_manager.cc:1347] Generated new cluster ID: 69b6aee8ff024afeb56cf47e9916a92b
I20260928 16:02:31.131309 12716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:31.158684 12716 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:31.159121 12716 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:31.167348 12716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a52f6256b4a44cf4b6b63a1affd29a00: Generated new TSK 0
I20260928 16:02:31.168356 12716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:31.175668 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12630
W20260928 16:02:31.176781 12692 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48177
W20260928 16:02:31.177482 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12630: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:31.177631 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12630
2026-09-28T16:02:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1859 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:31 dist-test-slave-9cqd krb5kdc[12728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:31 dist-test-slave-9cqd krb5kdc[12728](info): set up 2 sockets
Sep 28 16:02:31 dist-test-slave-9cqd krb5kdc[12728](info): commencing operation
krb5kdc: starting...
W20260928 16:02:32.278206 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:32 dist-test-slave-9cqd krb5kdc[12728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:32.528208 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44267
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:44267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:32.855198 12741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:32.855489 12741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:32.855684 12741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:32.862871 12741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:32.862970 12741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:32.863034 12741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:32.863086 12741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:32.863183 12741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:32.873171 12741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:44267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:44267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:32.874507 12741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:32.875780 12741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:32.887143 12747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:32.887219 12746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:32.887202 12749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:32.888159 12741 server_base.cc:1034] running on GCE node
I20260928 16:02:32.888916 12741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:32.890254 12741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:32.891470 12741 hybrid_clock.cc:648] HybridClock initialized: now 1790611352891438 us; error 44 us; skew 500 ppm
I20260928 16:02:32.891803 12741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:32 dist-test-slave-9cqd krb5kdc[12728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611352, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:32.895267 12741 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:32.896672 12741 webserver.cc:460] Webserver started at http://127.22.125.254:45055/ using document root <none> and password file <none>
I20260928 16:02:32.897272 12741 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:32.897368 12741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:32.897603 12741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:32.900449 12741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7"
format_stamp: "Formatted at 2026-09-28 16:02:32 on dist-test-slave-9cqd"
server_key: "3f1b962f1d935572a3ed1ea0a690f5a7"
server_key_iv: "85e9a9f1d8cb7b829318695905112b90"
server_key_version: "encryptionkey@0"
I20260928 16:02:32.901095 12741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7"
format_stamp: "Formatted at 2026-09-28 16:02:32 on dist-test-slave-9cqd"
server_key: "3f1b962f1d935572a3ed1ea0a690f5a7"
server_key_iv: "85e9a9f1d8cb7b829318695905112b90"
server_key_version: "encryptionkey@0"
I20260928 16:02:32.905774 12741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 16:02:32.908913 12756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:32.910113 12741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 16:02:32.910257 12741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7"
format_stamp: "Formatted at 2026-09-28 16:02:32 on dist-test-slave-9cqd"
server_key: "3f1b962f1d935572a3ed1ea0a690f5a7"
server_key_iv: "85e9a9f1d8cb7b829318695905112b90"
server_key_version: "encryptionkey@0"
I20260928 16:02:32.910472 12741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:32.926442 12741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:32.930011 12741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:32.930238 12741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:32.948160 12741 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:44267
I20260928 16:02:32.948179 12808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:44267 every 8 connection(s)
I20260928 16:02:32.949563 12741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:32.951333 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12741
I20260928 16:02:32.951521 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:32.952107 23031 external_mini_cluster.cc:1428] Setting key 1531bc0537b97f5889c7348a8cbadf8d
I20260928 16:02:32.953837 12809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:32.964759 12809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7: Bootstrap starting.
Sep 28 16:02:32 dist-test-slave-9cqd krb5kdc[12728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:02:32.968492 12809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:32.969534 12809 log.cc:824] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:32.972815 12809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7: No bootstrap required, opened a new log
I20260928 16:02:32.973657 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:32.953854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46059 (local address 127.22.125.254:44267)
0928 16:02:32.954580 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:32.954592 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:32.954650 (+ 58us) server_negotiation.cc:407] Connection header received
0928 16:02:32.956288 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:32.956308 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:32.956680 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:32.957077 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:32.958523 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:32.959804 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:32.961091 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:32.961764 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:32.961989 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:32.966534 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:32.966542 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:32.966555 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:32.966598 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:32.969659 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:32.970493 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:32.970506 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:32.970513 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:32.970753 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:32.971161 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:32.971164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:32.971166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:32.971633 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:32.971866 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:32.972322 (+ 456us) server_negotiation.cc:299] Negotiation successful
0928 16:02:32.972609 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":327,"threads_started":1}
I20260928 16:02:32.977262 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:32.982185 12809 raft_consensus.cc:348] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44267 } }
I20260928 16:02:32.982662 12809 raft_consensus.cc:374] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:32.982810 12809 raft_consensus.cc:729] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df3cb01eb56e4ebca74cdc6b3f6ecff7, State: Initialized, Role: FOLLOWER
I20260928 16:02:32.983457 12809 consensus_queue.cc:260] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44267 } }
I20260928 16:02:32.983695 12809 raft_consensus.cc:388] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:32.983848 12809 raft_consensus.cc:482] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:32.984027 12809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:32.986253 12809 raft_consensus.cc:504] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44267 } }
I20260928 16:02:32.986729 12809 leader_election.cc:302] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df3cb01eb56e4ebca74cdc6b3f6ecff7; no voters:
I20260928 16:02:32.987445 12809 leader_election.cc:288] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:32.987550 12819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:32.988587 12819 raft_consensus.cc:686] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [term 1 LEADER]: Becoming Leader. State: Replica: df3cb01eb56e4ebca74cdc6b3f6ecff7, State: Running, Role: LEADER
I20260928 16:02:32.989343 12819 consensus_queue.cc:237] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44267 } }
I20260928 16:02:32.989658 12809 sys_catalog.cc:564] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:32.992550 12821 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df3cb01eb56e4ebca74cdc6b3f6ecff7. Latest consensus state: current_term: 1 leader_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44267 } } }
I20260928 16:02:32.993420 12821 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:32.993131 12820 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3cb01eb56e4ebca74cdc6b3f6ecff7" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 44267 } } }
I20260928 16:02:32.993952 12820 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:32.997641 12826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:32.999692 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:32.982276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44465 (local address 127.22.125.254:44267)
0928 16:02:32.982468 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:32.982474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:32.982957 (+ 483us) server_negotiation.cc:407] Connection header received
0928 16:02:32.983133 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:32.983138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:32.983216 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:32.983313 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:32.985000 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:32.985732 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:32.986997 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:32.987469 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:32.987618 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:32.990597 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:32.990606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:32.990614 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:32.990666 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:32.995486 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:32.996581 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:32.996588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:32.996594 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:32.996811 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:32.997307 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:32.997318 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:32.997324 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:32.997783 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:32.998857 (+ 1074us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:32.999028 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 16:02:32.999269 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":61}
I20260928 16:02:33.003567 12826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:33.011809 12826 catalog_manager.cc:1347] Generated new cluster ID: 19c222ff01b64ab4899b013c708fa269
I20260928 16:02:33.011948 12826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:33.036863 12826 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:33.038126 12826 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:33.056752 12826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df3cb01eb56e4ebca74cdc6b3f6ecff7: Generated new TSK 0
I20260928 16:02:33.057533 12826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:33.066481 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12741
I20260928 16:02:33.067845 12804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:33.161871 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12741
2026-09-28T16:02:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1981 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:33 dist-test-slave-9cqd krb5kdc[12842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:33 dist-test-slave-9cqd krb5kdc[12842](info): set up 2 sockets
Sep 28 16:02:33 dist-test-slave-9cqd krb5kdc[12842](info): commencing operation
krb5kdc: starting...
W20260928 16:02:34.270406 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:34 dist-test-slave-9cqd krb5kdc[12842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:34.514863 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39491
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:34.845409 12855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:34.845774 12855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:34.845954 12855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:34.853612 12855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:34.853801 12855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:34.854015 12855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:34.854169 12855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:34.854272 12855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:34.865450 12855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:39491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:34.866909 12855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:34.868378 12855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:34.881047 12860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:34.880985 12861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:34.881095 12863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:34.882129 12855 server_base.cc:1034] running on GCE node
I20260928 16:02:34.882918 12855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:34.884446 12855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:34.885692 12855 hybrid_clock.cc:648] HybridClock initialized: now 1790611354885645 us; error 33 us; skew 500 ppm
I20260928 16:02:34.886101 12855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:34 dist-test-slave-9cqd krb5kdc[12842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611354, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:34.889582 12855 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:34.890965 12855 webserver.cc:460] Webserver started at http://127.22.125.254:43269/ using document root <none> and password file <none>
I20260928 16:02:34.891631 12855 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:34.891794 12855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:34.892153 12855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:34.894995 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "226802f8942a49d48f3d16db83796a48"
format_stamp: "Formatted at 2026-09-28 16:02:34 on dist-test-slave-9cqd"
server_key: "0e8d4499976bdd80baa4e44e94077fa8"
server_key_iv: "be541c628b05b4ff005890448ae76edb"
server_key_version: "encryptionkey@0"
I20260928 16:02:34.895725 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "226802f8942a49d48f3d16db83796a48"
format_stamp: "Formatted at 2026-09-28 16:02:34 on dist-test-slave-9cqd"
server_key: "0e8d4499976bdd80baa4e44e94077fa8"
server_key_iv: "be541c628b05b4ff005890448ae76edb"
server_key_version: "encryptionkey@0"
I20260928 16:02:34.900617 12855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260928 16:02:34.903805 12870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:34.905686 12855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:02:34.905848 12855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "226802f8942a49d48f3d16db83796a48"
format_stamp: "Formatted at 2026-09-28 16:02:34 on dist-test-slave-9cqd"
server_key: "0e8d4499976bdd80baa4e44e94077fa8"
server_key_iv: "be541c628b05b4ff005890448ae76edb"
server_key_version: "encryptionkey@0"
I20260928 16:02:34.906102 12855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:34.930975 12855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:34.935240 12855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:34.935479 12855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:34.953630 12855 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:39491
I20260928 16:02:34.953666 12921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:39491 every 8 connection(s)
I20260928 16:02:34.954892 12855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:34.958601 12922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:34.960434 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12855
I20260928 16:02:34.960623 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:34.961164 23031 external_mini_cluster.cc:1428] Setting key 24a76eb3bd41f7aa908ece64be2d5582
I20260928 16:02:34.971202 12922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48: Bootstrap starting.
I20260928 16:02:34.973548 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:34.962974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44229 (local address 127.22.125.254:39491)
0928 16:02:34.963704 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:34.963718 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:34.963760 (+ 42us) server_negotiation.cc:407] Connection header received
0928 16:02:34.965348 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:34.965369 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:34.965747 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:34.966169 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:34.966682 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:34.967917 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:34.969248 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:34.969916 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:34.970102 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:34.971321 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:34.971334 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:34.971354 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:34.971885 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:34.971957 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:34.971981 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:34.972131 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:34.972355 (+ 224us) server_negotiation.cc:299] Negotiation successful
0928 16:02:34.972619 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":341,"threads_started":1}
I20260928 16:02:34.974495 12922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:34.975864 12922 log.cc:824] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:34.977463 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:34.979067 12922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48: No bootstrap required, opened a new log
I20260928 16:02:34.988760 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:34.982532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45779 (local address 127.22.125.254:39491)
0928 16:02:34.982702 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:34.982707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:34.983046 (+ 339us) server_negotiation.cc:407] Connection header received
0928 16:02:34.983184 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:34.983189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:34.983266 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:34.983389 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:34.983830 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:34.984978 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:34.986237 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:34.986923 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:34.987055 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:34.988009 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:34.988018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:34.988024 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:34.988066 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:34.988110 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:34.988130 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:34.988216 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:34.988359 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 16:02:34.988442 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 16:02:34.988850 12922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226802f8942a49d48f3d16db83796a48" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39491 } }
I20260928 16:02:34.989259 12922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:34.989406 12922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226802f8942a49d48f3d16db83796a48, State: Initialized, Role: FOLLOWER
I20260928 16:02:34.989943 12922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226802f8942a49d48f3d16db83796a48" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39491 } }
I20260928 16:02:34.990188 12922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:34.990347 12922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:34.990514 12922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:34.992691 12922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226802f8942a49d48f3d16db83796a48" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39491 } }
I20260928 16:02:34.993170 12922 leader_election.cc:302] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 226802f8942a49d48f3d16db83796a48; no voters:
I20260928 16:02:34.993804 12922 leader_election.cc:288] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:34.993932 12932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:34.994727 12932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [term 1 LEADER]: Becoming Leader. State: Replica: 226802f8942a49d48f3d16db83796a48, State: Running, Role: LEADER
I20260928 16:02:34.995275 12932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226802f8942a49d48f3d16db83796a48" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39491 } }
I20260928 16:02:34.996222 12922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:34.998024 12933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "226802f8942a49d48f3d16db83796a48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226802f8942a49d48f3d16db83796a48" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39491 } } }
I20260928 16:02:34.998369 12933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:34.998139 12934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 226802f8942a49d48f3d16db83796a48. Latest consensus state: current_term: 1 leader_uuid: "226802f8942a49d48f3d16db83796a48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226802f8942a49d48f3d16db83796a48" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 39491 } } }
I20260928 16:02:34.998575 12934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:34.999295 12937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:35.005635 12937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:35.020689 12937 catalog_manager.cc:1347] Generated new cluster ID: fcfbe6f23c484ebe8321a54005edd819
I20260928 16:02:35.020808 12937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:35.035694 12937 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:35.037197 12937 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:35.054565 12937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 226802f8942a49d48f3d16db83796a48: Generated new TSK 0
I20260928 16:02:35.055667 12937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:35.063158 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12855
W20260928 16:02:35.064396 12917 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44229
W20260928 16:02:35.065068 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12855: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:35.065233 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12855
2026-09-28T16:02:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1905 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:35 dist-test-slave-9cqd krb5kdc[12953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:35 dist-test-slave-9cqd krb5kdc[12953](info): set up 2 sockets
Sep 28 16:02:35 dist-test-slave-9cqd krb5kdc[12953](info): commencing operation
krb5kdc: starting...
W20260928 16:02:36.162418 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:36 dist-test-slave-9cqd krb5kdc[12953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:36.405593 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34161
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:34161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:36.734764 12966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:36.735031 12966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:36.735131 12966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:36.741946 12966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:36.742043 12966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:36.742092 12966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:36.742177 12966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:36.742267 12966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:36.752054 12966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:34161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:34161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:36.753334 12966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:36.754658 12966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:36.767097 12972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:36.767000 12973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:36.767061 12975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:36.767874 12966 server_base.cc:1034] running on GCE node
I20260928 16:02:36.768647 12966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:36.769999 12966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:36.771224 12966 hybrid_clock.cc:648] HybridClock initialized: now 1790611356771185 us; error 32 us; skew 500 ppm
I20260928 16:02:36.771580 12966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:36 dist-test-slave-9cqd krb5kdc[12953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611356, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:36.778327 12966 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:36.779736 12966 webserver.cc:460] Webserver started at http://127.22.125.254:40953/ using document root <none> and password file <none>
I20260928 16:02:36.780476 12966 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:36.780616 12966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:36.780920 12966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:36.783793 12966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "6d1a85884ae54e90b2f52ea11d057047"
format_stamp: "Formatted at 2026-09-28 16:02:36 on dist-test-slave-9cqd"
server_key: "7568a8a7b2b98409c79795a309934ef5"
server_key_iv: "953be439e1fce162e8b91871bbb8f69c"
server_key_version: "encryptionkey@0"
I20260928 16:02:36.784592 12966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "6d1a85884ae54e90b2f52ea11d057047"
format_stamp: "Formatted at 2026-09-28 16:02:36 on dist-test-slave-9cqd"
server_key: "7568a8a7b2b98409c79795a309934ef5"
server_key_iv: "953be439e1fce162e8b91871bbb8f69c"
server_key_version: "encryptionkey@0"
I20260928 16:02:36.789528 12966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 16:02:36.792811 12982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:36.794061 12966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:02:36.794242 12966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "6d1a85884ae54e90b2f52ea11d057047"
format_stamp: "Formatted at 2026-09-28 16:02:36 on dist-test-slave-9cqd"
server_key: "7568a8a7b2b98409c79795a309934ef5"
server_key_iv: "953be439e1fce162e8b91871bbb8f69c"
server_key_version: "encryptionkey@0"
I20260928 16:02:36.794430 12966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:36.800132 12966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:36.804471 12966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:36.804771 12966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:36.823081 12966 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:34161
I20260928 16:02:36.823109 13033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:34161 every 8 connection(s)
I20260928 16:02:36.824548 12966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:36.826925 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 12966
I20260928 16:02:36.827105 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:36.827661 23031 external_mini_cluster.cc:1428] Setting key 5f42828d9893ae23edbdbf8923b964df
I20260928 16:02:36.828715 13034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:36.835263 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:36.829474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57213 (local address 127.22.125.254:34161)
0928 16:02:36.830225 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:36.830236 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:36.830275 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:02:36.831791 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:36.831811 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:36.832217 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:36.832612 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:36.833641 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:36.833647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:36.833658 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:36.833671 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:36.833748 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:36.833767 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:36.834007 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:36.834170 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 16:02:36.834358 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":314,"threads_started":1}
I20260928 16:02:36.838773 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:36.840876 13034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047: Bootstrap starting.
I20260928 16:02:36.844457 13034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:36.845607 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:36.842995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45293 (local address 127.22.125.254:34161)
0928 16:02:36.843173 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:36.843179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:36.843723 (+ 544us) server_negotiation.cc:407] Connection header received
0928 16:02:36.843790 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:36.843795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:36.843852 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:36.843960 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:36.844927 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:36.844934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:36.844945 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 16:02:36.844968 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:36.845022 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 16:02:36.845042 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:36.845170 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:36.845256 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 16:02:36.845318 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 16:02:36.845878 13034 log.cc:824] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:36.848809 13034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047: No bootstrap required, opened a new log
I20260928 16:02:36.856014 13034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1a85884ae54e90b2f52ea11d057047" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34161 } }
I20260928 16:02:36.856518 13034 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 16:02:36.856638 13034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d1a85884ae54e90b2f52ea11d057047, State: Initialized, Role: FOLLOWER
I20260928 16:02:36.857446 13034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1a85884ae54e90b2f52ea11d057047" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34161 } }
I20260928 16:02:36.857658 13034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 16:02:36.857759 13034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 16:02:36.857884 13034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:36.859949 13034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1a85884ae54e90b2f52ea11d057047" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34161 } }
I20260928 16:02:36.860481 13034 leader_election.cc:302] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d1a85884ae54e90b2f52ea11d057047; no voters:
I20260928 16:02:36.861140 13034 leader_election.cc:288] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 16:02:36.861280 13044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:36.861975 13044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [term 1 LEADER]: Becoming Leader. State: Replica: 6d1a85884ae54e90b2f52ea11d057047, State: Running, Role: LEADER
I20260928 16:02:36.862485 13044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1a85884ae54e90b2f52ea11d057047" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34161 } }
I20260928 16:02:36.863253 13034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:36.865633 13045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d1a85884ae54e90b2f52ea11d057047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1a85884ae54e90b2f52ea11d057047" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34161 } } }
I20260928 16:02:36.866252 13045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:36.865713 13046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d1a85884ae54e90b2f52ea11d057047. Latest consensus state: current_term: 1 leader_uuid: "6d1a85884ae54e90b2f52ea11d057047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1a85884ae54e90b2f52ea11d057047" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 34161 } } }
I20260928 16:02:36.867056 13046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:36.868418 13051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:36.872973 13051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:36.881614 13051 catalog_manager.cc:1347] Generated new cluster ID: 8218ee66670a44beab2023c9479d4e99
I20260928 16:02:36.881723 13051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:36.899734 13051 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:36.900235 13051 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:36.911821 13051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d1a85884ae54e90b2f52ea11d057047: Generated new TSK 0
I20260928 16:02:36.912671 13051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:36.924222 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12966
W20260928 16:02:36.925285 13029 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57213
W20260928 16:02:36.925959 23031 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 12966: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 16:02:36.926136 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 12966
2026-09-28T16:02:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1861 ms)
[----------] 18 tests from AuthTokenIssuingTest (34317 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:37 dist-test-slave-9cqd krb5kdc[13065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:37 dist-test-slave-9cqd krb5kdc[13065](info): set up 2 sockets
Sep 28 16:02:37 dist-test-slave-9cqd krb5kdc[13065](info): commencing operation
krb5kdc: starting...
W20260928 16:02:38.026366 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:38 dist-test-slave-9cqd krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:38.271564 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35407
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:35407,127.22.125.253:40687,127.22.125.252:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:38.600821 13078 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:02:38.601164 13078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:38.601284 13078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:38.601418 13078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:38.608589 13078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:38.608664 13078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:38.608790 13078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:38.608861 13078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:38.608985 13078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:38.618527 13078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:35407,127.22.125.253:40687,127.22.125.252:36983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:35407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:38.619724 13078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:38.621078 13078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:38.632629 13083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:38.632671 13084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:38.632720 13086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:38.633958 13078 server_base.cc:1034] running on GCE node
I20260928 16:02:38.634735 13078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:38.636157 13078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:38.637395 13078 hybrid_clock.cc:648] HybridClock initialized: now 1790611358637330 us; error 50 us; skew 500 ppm
I20260928 16:02:38.637720 13078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:38 dist-test-slave-9cqd krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:38.640846 13078 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:38.642198 13078 webserver.cc:460] Webserver started at http://127.22.125.254:40697/ using document root <none> and password file <none>
I20260928 16:02:38.642861 13078 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:38.642992 13078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:38.643314 13078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:38.646181 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "7681f5cf22104c1e8fa27d859b3ff18b"
format_stamp: "Formatted at 2026-09-28 16:02:38 on dist-test-slave-9cqd"
server_key: "5aa1bff7715e4d5632f6461810fec1e7"
server_key_iv: "38a4a4746a0298b5174b5b01ccef69d6"
server_key_version: "encryptionkey@0"
I20260928 16:02:38.646961 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "7681f5cf22104c1e8fa27d859b3ff18b"
format_stamp: "Formatted at 2026-09-28 16:02:38 on dist-test-slave-9cqd"
server_key: "5aa1bff7715e4d5632f6461810fec1e7"
server_key_iv: "38a4a4746a0298b5174b5b01ccef69d6"
server_key_version: "encryptionkey@0"
I20260928 16:02:38.651790 13078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260928 16:02:38.655086 13093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:38.656380 13078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 16:02:38.656550 13078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "7681f5cf22104c1e8fa27d859b3ff18b"
format_stamp: "Formatted at 2026-09-28 16:02:38 on dist-test-slave-9cqd"
server_key: "5aa1bff7715e4d5632f6461810fec1e7"
server_key_iv: "38a4a4746a0298b5174b5b01ccef69d6"
server_key_version: "encryptionkey@0"
I20260928 16:02:38.656760 13078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:38.675441 13078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:38.679610 13078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:38.679880 13078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:38.697989 13078 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:35407
I20260928 16:02:38.698015 13145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:35407 every 8 connection(s)
I20260928 16:02:38.699298 13078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:38.702916 13146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:38.703451 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 13078
I20260928 16:02:38.703667 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:38.704535 23031 external_mini_cluster.cc:1428] Setting key 708b95dd5b74677c18dc6c323ad4ebcd
I20260928 16:02:38.712980 13146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:38 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.253@KRBTEST.COM" created.
I20260928 16:02:38.735472 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.717028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35407 (local address 127.0.0.1:36524)
0928 16:02:38.717758 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.717802 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:38.719911 (+ 2109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:38.720756 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:38.720762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:38.721450 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:38.721770 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:38.721788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:38.723343 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:38.723348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:38.724453 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:38.724465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:38.725370 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:38.725377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:38.725569 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:38.726257 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:38.726278 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:38.728534 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:38.731718 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:38.731731 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:38.731741 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:38.732207 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:38.732611 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:38.732617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:38.732620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:38.732779 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:38.733729 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:38.733735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:38.733923 (+ 188us) client_negotiation.cc:769] Sending connection context
0928 16:02:38.734125 (+ 202us) client_negotiation.cc:240] Negotiation successful
0928 16:02:38.734469 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":307,"threads_started":1}
I20260928 16:02:38.735561 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.718235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.22.125.254:35407)
0928 16:02:38.719060 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:38.719074 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:38.719126 (+ 52us) server_negotiation.cc:407] Connection header received
0928 16:02:38.720320 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:38.720326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:38.720537 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:38.720738 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:38.721985 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:38.723167 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:38.724590 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:38.725134 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:38.725336 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:38.728771 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:38.728779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:38.728789 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:38.728822 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:38.731592 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:38.732344 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:38.732351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:38.732354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:38.732490 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:38.732923 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:38.732927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:38.732929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:38.733566 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:38.733775 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:38.734345 (+ 570us) server_negotiation.cc:299] Negotiation successful
0928 16:02:38.734632 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":422,"threads_started":1}
I20260928 16:02:38.738533 13146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:38.739624 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.739266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42130)
0928 16:02:38.739429 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.739529 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260928 16:02:38.739909 13094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
I20260928 16:02:38.741459 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.741110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42140)
0928 16:02:38.741251 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.741338 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260928 16:02:38.741659 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:40687: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab.
Entry for principal kudu/127.22.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab.
I20260928 16:02:38.761428 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:40687
--webserver_interface=127.22.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab
--principal=kudu/127.22.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:35407,127.22.125.253:40687,127.22.125.252:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:02:38.791020 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } attempt: 1
I20260928 16:02:38.792296 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.791748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42150)
0928 16:02:38.791967 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.792131 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260928 16:02:38.793444 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.793040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42154)
0928 16:02:38.793200 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.793287 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260928 16:02:38.793678 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:40687: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
I20260928 16:02:38.861923 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } attempt: 2
I20260928 16:02:38.863248 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.862641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42162)
0928 16:02:38.862960 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.863074 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260928 16:02:38.864336 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.863897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42172)
0928 16:02:38.864077 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.864181 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:02:38.864526 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:40687: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
I20260928 16:02:38.955838 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } attempt: 3
I20260928 16:02:38.957316 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.956693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42182)
0928 16:02:38.956972 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.957111 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260928 16:02:38.958439 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:38.958029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42192)
0928 16:02:38.958192 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:38.958277 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 16:02:38.958663 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:40687: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
W20260928 16:02:39.091329 13153 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:02:39.091684 13153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:39.091807 13153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:39.091945 13153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:39.099224 13153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:39.099313 13153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:39.099447 13153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:39.099524 13153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:39.099638 13153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:39.102053 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } attempt: 4
I20260928 16:02:39.103334 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.102706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42196)
0928 16:02:39.103052 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.103163 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260928 16:02:39.104400 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.103990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42208)
0928 16:02:39.104164 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.104257 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260928 16:02:39.104593 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:40687: Network error: Client connection negotiation failed: client connection to 127.22.125.253:40687: connect: Connection refused (error 111)
I20260928 16:02:39.110407 13153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:35407,127.22.125.253:40687,127.22.125.252:36983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab
--principal=kudu/127.22.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:40687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:39.111682 13153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:39.113050 13153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:39.125025 13161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:39.125101 13159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:39.124993 13158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:39.133504 13153 server_base.cc:1034] running on GCE node
I20260928 16:02:39.134331 13153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:39.135794 13153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:39.137061 13153 hybrid_clock.cc:648] HybridClock initialized: now 1790611359136979 us; error 62 us; skew 500 ppm
I20260928 16:02:39.137400 13153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:39.140570 13153 init.cc:377] Logged in from keytab as kudu/127.22.125.253@KRBTEST.COM (short username kudu)
I20260928 16:02:39.142120 13153 webserver.cc:460] Webserver started at http://127.22.125.253:36983/ using document root <none> and password file <none>
I20260928 16:02:39.142845 13153 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:39.142977 13153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:39.143299 13153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:39.146387 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data/instance:
uuid: "c126229749f744fdac2770fa4cab5329"
format_stamp: "Formatted at 2026-09-28 16:02:39 on dist-test-slave-9cqd"
server_key: "ad44bb3fea87877ea43c16011c616576"
server_key_iv: "4546b53118eb33a963053f34bea1c7ac"
server_key_version: "encryptionkey@0"
I20260928 16:02:39.147161 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/wal/instance:
uuid: "c126229749f744fdac2770fa4cab5329"
format_stamp: "Formatted at 2026-09-28 16:02:39 on dist-test-slave-9cqd"
server_key: "ad44bb3fea87877ea43c16011c616576"
server_key_iv: "4546b53118eb33a963053f34bea1c7ac"
server_key_version: "encryptionkey@0"
I20260928 16:02:39.152289 13153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 16:02:39.155988 13168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:39.157328 13153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 16:02:39.157512 13153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/wal
uuid: "c126229749f744fdac2770fa4cab5329"
format_stamp: "Formatted at 2026-09-28 16:02:39 on dist-test-slave-9cqd"
server_key: "ad44bb3fea87877ea43c16011c616576"
server_key_iv: "4546b53118eb33a963053f34bea1c7ac"
server_key_version: "encryptionkey@0"
I20260928 16:02:39.157733 13153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:39.177032 13153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:39.180841 13153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:39.181116 13153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:39.199515 13153 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.253:40687
I20260928 16:02:39.199532 13219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.253:40687 every 8 connection(s)
I20260928 16:02:39.200907 13153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
I20260928 16:02:39.204730 13220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:39.206688 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 13153
I20260928 16:02:39.206954 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-1/wal/instance
I20260928 16:02:39.207399 23031 external_mini_cluster.cc:1428] Setting key 876e9115c0adad548e163c2b364b4f5c
I20260928 16:02:39.211336 13220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.252@KRBTEST.COM; defaulting to no policy
I20260928 16:02:39.233171 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.213877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.22.125.254:35407)
0928 16:02:39.214055 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.214059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.215477 (+ 1418us) server_negotiation.cc:407] Connection header received
0928 16:02:39.217710 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.217719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.217816 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.217929 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.219362 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.220561 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.222040 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.222789 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.222932 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.226290 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.226299 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.226307 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.226353 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.230268 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.230947 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.230953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.230957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.231111 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.231501 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.231508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.231515 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.231935 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.232125 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.232772 (+ 647us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.232945 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:02:39.234202 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.214458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35407 (local address 127.0.0.1:36532)
0928 16:02:39.215265 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.215327 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.217434 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.218121 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.218129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.218630 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.219175 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.219191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.220721 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.220725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.221881 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.221894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.222986 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.222990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.223188 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.223874 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.223896 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.226032 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.230401 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.230407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.230418 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.230824 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.231224 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.231227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.231230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.231388 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.232111 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.232116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.232453 (+ 337us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.232682 (+ 229us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.233029 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":307,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.252@KRBTEST.COM" created.
I20260928 16:02:39.235978 13220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:02:39.252094 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.236748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42218)
0928 16:02:39.236942 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.236963 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.237070 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.237988 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.237995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.238481 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.238689 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.238703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.240025 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.240033 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.241569 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.241587 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.242390 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.242400 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.242617 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.243231 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.243258 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.245575 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.249812 (+ 4237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.249815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.249818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.250159 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.250566 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.250569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.250571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.250701 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.251404 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.251407 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.251518 (+ 111us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.251622 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.251797 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260928 16:02:39.252584 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.236949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.22.125.253:40687)
0928 16:02:39.237580 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.237585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.237610 (+ 25us) server_negotiation.cc:407] Connection header received
0928 16:02:39.237649 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.237654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.237826 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.237932 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.238864 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.239885 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.241704 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.242216 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.242309 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.245799 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.245815 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.245832 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.245879 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.249682 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.250293 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.250305 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.250311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.250460 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.250812 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.250820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.250826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.251284 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.251503 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.251804 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.252099 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":359,"threads_started":1}
I20260928 16:02:39.254920 13220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:39.256116 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.255598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36160)
0928 16:02:39.255785 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.255909 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260928 16:02:39.256397 13171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
I20260928 16:02:39.257928 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.257589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36176)
0928 16:02:39.257744 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.257834 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 16:02:39.258126 13220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab.
Entry for principal kudu/127.22.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab.
I20260928 16:02:39.267114 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:36983
--webserver_interface=127.22.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab
--principal=kudu/127.22.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:41683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:35407,127.22.125.253:40687,127.22.125.252:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:02:39.307416 13220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 1
I20260928 16:02:39.308563 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.308121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36178)
0928 16:02:39.308307 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.308417 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 16:02:39.309475 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.309161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36192)
0928 16:02:39.309316 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.309382 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 16:02:39.309701 13220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
I20260928 16:02:39.377895 13220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 2
I20260928 16:02:39.379092 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.378530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36194)
0928 16:02:39.378768 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.378893 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260928 16:02:39.380165 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.379729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36200)
0928 16:02:39.379890 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.379974 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 16:02:39.380380 13220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
I20260928 16:02:39.403846 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } attempt: 5
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:02:39.417927 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.404786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42234 (local address 127.22.125.253:40687)
0928 16:02:39.404957 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.404965 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.405062 (+ 97us) server_negotiation.cc:407] Connection header received
0928 16:02:39.405292 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.405307 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.405386 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.405492 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.406341 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.407041 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.408308 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.408767 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.408901 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.412405 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.412410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.412413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.412447 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.415236 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.415916 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.415924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.415932 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.416134 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.416528 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.416535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.416539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.416825 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.416994 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.417363 (+ 369us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.417589 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:02:39.417939 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.404688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42234)
0928 16:02:39.404942 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.404965 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.405143 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.405526 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.405531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.405964 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.406172 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.406182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.407179 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.407186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.408152 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.408165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.408926 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.408932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.409146 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.409825 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.409843 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.412212 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.415407 (+ 3195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.415418 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.415422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.415782 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.416253 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.416258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.416262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.416415 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.416955 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.416959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.417123 (+ 164us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.417266 (+ 143us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.417586 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260928 16:02:39.419139 13146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:39.420346 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.419925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36216)
0928 16:02:39.420137 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.420211 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 16:02:39.421782 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.421475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36228)
0928 16:02:39.421619 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.421693 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260928 16:02:39.422031 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
I20260928 16:02:39.471642 13220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 3
I20260928 16:02:39.472998 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.472404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36242)
0928 16:02:39.472651 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.472789 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260928 16:02:39.473275 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 1
I20260928 16:02:39.474222 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.473728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36246)
0928 16:02:39.474000 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.474089 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260928 16:02:39.474342 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.473809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36254)
0928 16:02:39.474062 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.474157 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260928 16:02:39.474483 13220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
I20260928 16:02:39.475423 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.475017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36258)
0928 16:02:39.475191 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.475281 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 16:02:39.475665 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
I20260928 16:02:39.543937 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 2
I20260928 16:02:39.545264 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.544675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36268)
0928 16:02:39.544946 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.545068 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 16:02:39.546372 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.545931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36280)
0928 16:02:39.546113 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.546224 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260928 16:02:39.546639 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
W20260928 16:02:39.593212 13227 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:02:39.593587 13227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:39.593715 13227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:39.593858 13227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:39.600965 13227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:39.601081 13227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:39.601187 13227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:39.601293 13227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:39.601387 13227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:39.610994 13227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:41683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:35407,127.22.125.253:40687,127.22.125.252:36983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab
--principal=kudu/127.22.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:36983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:39.612291 13227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:39.613560 13227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 16:02:39.617707 13220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 4
I20260928 16:02:39.619077 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.618386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36284)
0928 16:02:39.618710 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.618852 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260928 16:02:39.620316 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.619968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36296)
0928 16:02:39.620155 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.620219 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260928 16:02:39.620508 13220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
W20260928 16:02:39.625533 13233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:39.625643 13232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:39.625682 13235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:39.626402 13227 server_base.cc:1034] running on GCE node
I20260928 16:02:39.627041 13227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:39.628396 13227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:39.629621 13227 hybrid_clock.cc:648] HybridClock initialized: now 1790611359629563 us; error 33 us; skew 500 ppm
I20260928 16:02:39.629943 13227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:39.633090 13227 init.cc:377] Logged in from keytab as kudu/127.22.125.252@KRBTEST.COM (short username kudu)
I20260928 16:02:39.634449 13227 webserver.cc:460] Webserver started at http://127.22.125.252:44995/ using document root <none> and password file <none>
I20260928 16:02:39.635289 13227 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:39.635385 13227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:39.635646 13227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:39.638523 13227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data/instance:
uuid: "045c81fba9624184a9d919ca1c9a201e"
format_stamp: "Formatted at 2026-09-28 16:02:39 on dist-test-slave-9cqd"
server_key: "844673d1ac2045347cb8f7d8d584ea99"
server_key_iv: "8a1ae7507e6e1a90f644e68d4d275032"
server_key_version: "encryptionkey@0"
I20260928 16:02:39.639258 13227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/wal/instance:
uuid: "045c81fba9624184a9d919ca1c9a201e"
format_stamp: "Formatted at 2026-09-28 16:02:39 on dist-test-slave-9cqd"
server_key: "844673d1ac2045347cb8f7d8d584ea99"
server_key_iv: "8a1ae7507e6e1a90f644e68d4d275032"
server_key_version: "encryptionkey@0"
I20260928 16:02:39.644259 13227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260928 16:02:39.647748 13242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:39.649180 13227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:39.649328 13227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/wal
uuid: "045c81fba9624184a9d919ca1c9a201e"
format_stamp: "Formatted at 2026-09-28 16:02:39 on dist-test-slave-9cqd"
server_key: "844673d1ac2045347cb8f7d8d584ea99"
server_key_iv: "8a1ae7507e6e1a90f644e68d4d275032"
server_key_version: "encryptionkey@0"
I20260928 16:02:39.649505 13227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:39.652843 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 3
I20260928 16:02:39.653927 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.653402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36300)
0928 16:02:39.653615 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.653721 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260928 16:02:39.655092 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.654720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36310)
0928 16:02:39.654921 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.654996 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260928 16:02:39.655406 13146 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:36983: Network error: Client connection negotiation failed: client connection to 127.22.125.252:36983: connect: Connection refused (error 111)
I20260928 16:02:39.685220 13227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:39.689633 13227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:39.689906 13227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:39.708611 13227 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.252:36983
I20260928 16:02:39.708634 13294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.252:36983 every 8 connection(s)
I20260928 16:02:39.709946 13227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
I20260928 16:02:39.713541 13295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:39.719871 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 13227
I20260928 16:02:39.720075 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790611005938867-23031-0/minicluster-data/master-2/wal/instance
I20260928 16:02:39.719708 13295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:39.720538 23031 external_mini_cluster.cc:1428] Setting key ae6c59fb860a6f1e5692ddf2ffaec0b3
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
I20260928 16:02:39.739148 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.722423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.22.125.254:35407)
0928 16:02:39.722641 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.722647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.723584 (+ 937us) server_negotiation.cc:407] Connection header received
0928 16:02:39.725229 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.725234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.725321 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.725427 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.726648 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.727354 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.728927 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.729361 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.729497 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.733134 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.733141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.733146 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.733189 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.735923 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.736879 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.736882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.736885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.737009 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.737479 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.737482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.737484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.737718 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.737916 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.738590 (+ 674us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.738840 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 16:02:39.740208 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.722508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:35407 (local address 127.0.0.1:36548)
0928 16:02:39.723446 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.723478 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.724998 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.725485 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.725491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.726138 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.726486 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.726503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.727520 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.727524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.728751 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.728762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.729591 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.729598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.729861 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.730599 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.730628 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.732942 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.736078 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.736095 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.736106 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.736754 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.737122 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.737126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.737129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.737362 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.737888 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.737894 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.738161 (+ 267us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.738459 (+ 298us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.738862 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":37,"thread_start_us":629,"threads_started":1}
I20260928 16:02:39.740617 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.722363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.22.125.252:36983)
0928 16:02:39.723210 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.723227 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.723276 (+ 49us) server_negotiation.cc:407] Connection header received
0928 16:02:39.724990 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.725011 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.725230 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.725450 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.726934 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.728131 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.729359 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.729955 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.730098 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.734373 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.734381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.734391 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.734421 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.737692 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.738342 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.738349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.738352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.738495 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.738892 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.738897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.738899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.739291 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.739576 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.739937 (+ 361us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.740214 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":581,"thread_start_us":345,"threads_started":1}
I20260928 16:02:39.742048 13295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:02:39.755690 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.742682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:40687 (local address 127.0.0.1:42242)
0928 16:02:39.742870 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.742888 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.742980 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.743307 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.743311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.743752 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.743939 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.743950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.745034 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.745041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.746089 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.746101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.746861 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.746867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.747078 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.747589 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.747611 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.749812 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.753394 (+ 3582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.753399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.753403 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.753736 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.754139 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.754145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.754149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.754294 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.754806 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.754811 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.754952 (+ 141us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.755094 (+ 142us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.755345 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 16:02:39.755755 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.742800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.22.125.253:40687)
0928 16:02:39.742975 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.742981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.743000 (+ 19us) server_negotiation.cc:407] Connection header received
0928 16:02:39.743091 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.743096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.743164 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.743269 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.744118 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.744888 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.746237 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.746720 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.746814 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.749996 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.750000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.750003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.750035 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.753241 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.753864 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.753869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.753873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.754047 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.754407 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.754413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.754417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.754669 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.754832 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.755184 (+ 352us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.755401 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 16:02:39.756785 13295 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:39.758402 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.743747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36564 (local address 127.22.125.254:35407)
0928 16:02:39.743893 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.743897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.743911 (+ 14us) server_negotiation.cc:407] Connection header received
0928 16:02:39.744098 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.744103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.744175 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.744277 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.745485 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.746222 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.747725 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.748180 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.748271 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.752779 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.752783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.752786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.752816 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.755492 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.756374 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.756381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.756384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.756537 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.756974 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.756979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.756982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.757335 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.757619 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.757888 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.758114 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:02:39.771639 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.757558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36332)
0928 16:02:39.757730 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.757751 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.757849 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.758203 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.758208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.758653 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.758818 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.758829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.759980 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.759986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.761154 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.761171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.762149 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.762156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.762489 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.763111 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.763137 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.765025 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.768174 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.768373 (+ 199us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.768380 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.768910 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.769517 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.769562 (+ 45us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.769570 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.769808 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.770599 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.770604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.770761 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.770909 (+ 148us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.771238 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 16:02:39.771833 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.757771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.22.125.252:36983)
0928 16:02:39.757923 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.757928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.757942 (+ 14us) server_negotiation.cc:407] Connection header received
0928 16:02:39.757986 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.757990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.758055 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.758154 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.758971 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.759705 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.761285 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.761988 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.762171 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.765213 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.765219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.765224 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.765261 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.767968 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.769105 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.769175 (+ 70us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.769178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.769380 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.769964 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.769969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.769972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.770466 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.770638 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.771251 (+ 613us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.771472 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 16:02:39.776455 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.759984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.22.125.253:40687)
0928 16:02:39.760155 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.760162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.760244 (+ 82us) server_negotiation.cc:407] Connection header received
0928 16:02:39.760535 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.760540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.760623 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.760736 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.762371 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.763450 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.764825 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.765336 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.765468 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.770399 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.770405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.770409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.770445 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.773965 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.774593 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.774598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.774602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.774756 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.775114 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.775118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.775121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.775439 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.775702 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.775860 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.776119 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 16:02:39.777769 13295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: Bootstrap starting.
I20260928 16:02:39.781260 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:39.781553 13295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:39.782593 13295 log.cc:824] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:39.785449 13295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: No bootstrap required, opened a new log
I20260928 16:02:39.784009 13121 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7681f5cf22104c1e8fa27d859b3ff18b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36564
I20260928 16:02:39.792474 13295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:39.792801 13295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 045c81fba9624184a9d919ca1c9a201e, State: Initialized, Role: FOLLOWER
I20260928 16:02:39.793252 13295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:39.794183 13301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:39.794593 13301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:39.795296 13295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:02:39.804984 13312 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:02:39.805167 13312 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:02:39.802541 13227 master_runner.cc:186] Error getting master registration for 127.22.125.254:35407: OK, instance_id { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" instance_seqno: 1790611358692865 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:02:39.813781 13227 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:02:39.832631 13146 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 4
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611358, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
I20260928 16:02:39.849596 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.833398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36342)
0928 16:02:39.833679 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.833698 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.833831 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.834218 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.834222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.834778 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.835016 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.835030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.836496 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.836502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.837995 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.838010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.839068 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.839075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.839290 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.839971 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.839988 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.842457 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.846664 (+ 4207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.846674 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.846677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.847269 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.847673 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.847678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.847682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.847846 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.848557 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.848561 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.848736 (+ 175us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.848897 (+ 161us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.849192 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260928 16:02:39.849612 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.833460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.22.125.252:36983)
0928 16:02:39.833681 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.833688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.833776 (+ 88us) server_negotiation.cc:407] Connection header received
0928 16:02:39.833958 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.833963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.834047 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.834174 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.835176 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.836350 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.838156 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.838912 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.839041 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.842643 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.842651 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.842656 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.842700 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.846502 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.847405 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.847411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.847414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.847554 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.847959 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.847963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.847967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.848426 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.848611 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.849010 (+ 399us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.849242 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 16:02:39.855762 13146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b: Bootstrap starting.
I20260928 16:02:39.859822 13146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:39.861157 13146 log.cc:824] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:39.864507 13146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b: No bootstrap required, opened a new log
I20260928 16:02:39.872018 13146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:39.872452 13146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7681f5cf22104c1e8fa27d859b3ff18b, State: Initialized, Role: FOLLOWER
I20260928 16:02:39.872962 13146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:39.873936 13315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:39.874329 13315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:39.875151 13146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:39.875221 13121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 16:02:39.875429 13121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:39.877928 13121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:39.879622 13121 leader_election.cc:288] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [CANDIDATE]: Term 1 election: Requested vote from peers c126229749f744fdac2770fa4cab5329 (127.22.125.253:40687), 045c81fba9624184a9d919ca1c9a201e (127.22.125.252:36983)
I20260928 16:02:39.879995 13270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "045c81fba9624184a9d919ca1c9a201e"
I20260928 16:02:39.880532 13270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:39.883884 13270 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7681f5cf22104c1e8fa27d859b3ff18b in term 1.
I20260928 16:02:39.884483 13096 leader_election.cc:302] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 045c81fba9624184a9d919ca1c9a201e, 7681f5cf22104c1e8fa27d859b3ff18b; no voters:
I20260928 16:02:39.885597 13315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:39.915191 13315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [term 1 LEADER]: Becoming Leader. State: Replica: 7681f5cf22104c1e8fa27d859b3ff18b, State: Running, Role: LEADER
I20260928 16:02:39.915979 13315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:39.919817 13220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } attempt: 5
I20260928 16:02:39.921438 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.884521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.22.125.253:40687)
0928 16:02:39.884949 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.884955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.891863 (+ 6908us) server_negotiation.cc:407] Connection header received
0928 16:02:39.893111 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.893117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.893211 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.893300 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.897611 (+ 4311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.898819 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.903107 (+ 4288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.903763 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.903859 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.910816 (+ 6957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.910823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.910827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.910874 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.915658 (+ 4784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.916396 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.916402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.916405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.916559 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.917027 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.917032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.917036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.917418 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.917685 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.919460 (+ 1775us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.919705 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260928 16:02:39.922637 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.883808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.22.125.254:35407)
0928 16:02:39.883988 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.883994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.892862 (+ 8868us) server_negotiation.cc:407] Connection header received
0928 16:02:39.893325 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.893334 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.893440 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.893515 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.898408 (+ 4893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.899725 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.903722 (+ 3997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.904670 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.904805 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.910573 (+ 5768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.910582 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.910588 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.910633 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.917586 (+ 6953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.918892 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.918897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.918900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.919093 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.919734 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.919753 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.919757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.920130 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.920392 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.921273 (+ 881us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.921671 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:02:39.928997 13326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7681f5cf22104c1e8fa27d859b3ff18b. Latest consensus state: current_term: 1 leader_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:39.929543 13326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: This master's current role is: LEADER
W20260928 16:02:39.932542 13099 tablet.cc:2369] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:02:39.936656 13334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:39.939272 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.886315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.22.125.252:36983)
0928 16:02:39.886525 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.886534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.908585 (+ 22051us) server_negotiation.cc:407] Connection header received
0928 16:02:39.909894 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.909901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.909995 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.910071 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.912076 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.913248 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.920353 (+ 7105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.921081 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.921178 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.929704 (+ 8526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.929713 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.929719 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.929768 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.934767 (+ 4999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.936564 (+ 1797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.936572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.936577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.936738 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.937218 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.937226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.937229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.937561 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.937850 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.938349 (+ 499us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.938805 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":67}
W20260928 16:02:39.941962 13248 tablet.cc:2369] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 16:02:39 dist-test-slave-9cqd krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611359, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
I20260928 16:02:39.953284 13334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 16:02:39.957959 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.920380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:36983 (local address 127.0.0.1:36362)
0928 16:02:39.920707 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:39.920725 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:39.920817 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:39.936781 (+ 15964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:39.936788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:39.937263 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:39.937479 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.937493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.940876 (+ 3383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.940882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.942333 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:39.942350 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.943834 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.943840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:39.944141 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.944767 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:39.944784 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:39.947124 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:39.951419 (+ 4295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.951425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.951429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.951985 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.952438 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:39.952444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:39.952448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:39.952611 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:39.953622 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:39.953630 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:39.953917 (+ 287us) client_negotiation.cc:769] Sending connection context
0928 16:02:39.954643 (+ 726us) client_negotiation.cc:240] Negotiation successful
0928 16:02:39.956201 (+ 1558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208}
I20260928 16:02:39.957991 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:39.921008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.22.125.252:36983)
0928 16:02:39.936300 (+ 15292us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:39.936307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:39.936332 (+ 25us) server_negotiation.cc:407] Connection header received
0928 16:02:39.936385 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:39.936391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:39.936465 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:39.937650 (+ 1185us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:39.937920 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.940699 (+ 2779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.942985 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:39.943697 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:39.943821 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:39.947323 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:39.947330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:39.947335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:39.947370 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:39.951299 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.952158 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.952164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.952168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.952300 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:39.952705 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:39.952739 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:39.952746 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:39.953365 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:39.954246 (+ 881us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:39.954841 (+ 595us) server_negotiation.cc:299] Negotiation successful
0928 16:02:39.955752 (+ 911us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1456,"server-negotiator.queue_time_us":15132,"thread_start_us":15542,"threads_started":1}
I20260928 16:02:39.966588 13220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329: Bootstrap starting.
I20260928 16:02:39.971629 13220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:39.973112 13220 log.cc:824] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:39.974642 13270 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [term 1 FOLLOWER]: Refusing update from remote peer 7681f5cf22104c1e8fa27d859b3ff18b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:02:39.975735 13326 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [LEADER]: Connected to new peer: Peer: permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:02:39.983762 13220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329: No bootstrap required, opened a new log
I20260928 16:02:39.979295 13078 master_runner.cc:186] Error getting master registration for 127.22.125.253:40687: OK, instance_id { permanent_uuid: "c126229749f744fdac2770fa4cab5329" instance_seqno: 1790611359194247 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:02:40.000818 13220 raft_consensus.cc:348] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:40.001187 13078 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:02:40.001299 13220 raft_consensus.cc:729] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c126229749f744fdac2770fa4cab5329, State: Initialized, Role: FOLLOWER
I20260928 16:02:40.001889 13220 consensus_queue.cc:260] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } }
I20260928 16:02:40.003521 13301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7681f5cf22104c1e8fa27d859b3ff18b. Latest consensus state: current_term: 1 leader_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:40.003856 13301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:40.009150 13341 mvcc.cc:205] Tried to move back new op lower bound from 7334344130426474496 to 7334344130222886912. Current Snapshot: MvccSnapshot[applied={T|T < 7334344130426474496}]
I20260928 16:02:40.009954 13220 sys_catalog.cc:564] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:40.010010 13195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c126229749f744fdac2770fa4cab5329"
I20260928 16:02:40.010469 13195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:40.014626 13195 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7681f5cf22104c1e8fa27d859b3ff18b in term 1.
I20260928 16:02:40.015758 13194 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [term 1 FOLLOWER]: Refusing update from remote peer 7681f5cf22104c1e8fa27d859b3ff18b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:02:40.015709 13343 sys_catalog.cc:455] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:40.016134 13343 sys_catalog.cc:458] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:40.016686 13315 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:02:40.037604 13315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:40.038028 13315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:40.038106 13334 catalog_manager.cc:1347] Generated new cluster ID: e67701f97d4f403da1106f632101ea27
I20260928 16:02:40.038214 13334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:40.041477 13301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:40.041811 13301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:40.057099 13343 sys_catalog.cc:455] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7681f5cf22104c1e8fa27d859b3ff18b. Latest consensus state: current_term: 1 leader_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:40.057402 13343 sys_catalog.cc:458] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:40.057726 13351 sys_catalog.cc:455] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:40.058084 13351 sys_catalog.cc:458] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:40.057654 13334 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:40.060176 13334 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:40.060837 13315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7681f5cf22104c1e8fa27d859b3ff18b" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 35407 } } peers { permanent_uuid: "c126229749f744fdac2770fa4cab5329" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 40687 } } peers { permanent_uuid: "045c81fba9624184a9d919ca1c9a201e" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 36983 } } }
I20260928 16:02:40.061164 13315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:40.087692 13360 catalog_manager.cc:1259] Loaded cluster ID: e67701f97d4f403da1106f632101ea27
I20260928 16:02:40.087864 13360 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329: loading cluster ID for follower catalog manager: success
I20260928 16:02:40.089651 13334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7681f5cf22104c1e8fa27d859b3ff18b: Generated new TSK 0
I20260928 16:02:40.090615 13334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:40.091638 13360 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329: acquiring CA information for follower catalog manager: success
I20260928 16:02:40.093595 13360 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c126229749f744fdac2770fa4cab5329: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 16:02:40.222657 13224 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.203712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.22.125.253:40687)
0928 16:02:40.203935 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.203942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.220608 (+ 16666us) server_negotiation.cc:407] Connection header received
0928 16:02:40.220926 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.220933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.221036 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.221123 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.222420 (+ 1297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42284: BlockingRecv error: recv got EOF from 127.0.0.1:42284 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260928 16:02:40.224647 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.210081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.22.125.252:36983)
0928 16:02:40.210309 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.210315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.222153 (+ 11838us) server_negotiation.cc:407] Connection header received
0928 16:02:40.223010 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.223016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.223095 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.223169 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.224409 (+ 1240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36366: BlockingRecv error: recv got EOF from 127.0.0.1:36366 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260928 16:02:40.224757 13150 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.220846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.22.125.254:35407)
0928 16:02:40.221044 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.221055 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.223278 (+ 2223us) server_negotiation.cc:407] Connection header received
0928 16:02:40.223514 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.223519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.223627 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.223719 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.224526 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36586: BlockingRecv error: recv got EOF from 127.0.0.1:36586 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 16:02:40.275154 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.255986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.22.125.253:40687)
0928 16:02:40.256200 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.256207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.267032 (+ 10825us) server_negotiation.cc:407] Connection header received
0928 16:02:40.267597 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.267604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.267848 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.268000 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.268767 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.269983 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.271994 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.272696 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.272811 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.273586 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.274302 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.274426 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.274701 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.274805 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260928 16:02:40.316454 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.306517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36374 (local address 127.22.125.252:36983)
0928 16:02:40.306717 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.306724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.314225 (+ 7501us) server_negotiation.cc:407] Connection header received
0928 16:02:40.314790 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.314796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.314924 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.315006 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.316166 (+ 1160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36374: BlockingRecv error: recv got EOF from 127.0.0.1:36374 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 16:02:40.368327 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.349936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.22.125.253:40687)
0928 16:02:40.350169 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.350176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.361301 (+ 11125us) server_negotiation.cc:407] Connection header received
0928 16:02:40.361619 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.361625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.361765 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.361835 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.362670 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.363780 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.365871 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.366621 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.366741 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.367373 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.367538 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.367608 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.367876 (+ 268us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.367961 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260928 16:02:40.400847 13294 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.22.125.252:36983: Invalid argument: socket address is ambiguous
W20260928 16:02:40.410712 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.401203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.22.125.252:36983)
0928 16:02:40.401404 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.401410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.408792 (+ 7382us) server_negotiation.cc:407] Connection header received
0928 16:02:40.409256 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.409262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.409379 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.409455 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.410478 (+ 1023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36380: BlockingRecv error: recv got EOF from 127.0.0.1:36380 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:02:40.467330 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.448410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.22.125.253:40687)
0928 16:02:40.448610 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.448617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.460890 (+ 12273us) server_negotiation.cc:407] Connection header received
0928 16:02:40.461234 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.461240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.461328 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.461415 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.462248 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.463501 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.465254 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.465810 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.465899 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.466505 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.466650 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.466708 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.466920 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.467013 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260928 16:02:40.477991 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.474715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.22.125.252:36983)
0928 16:02:40.474897 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.474904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.476544 (+ 1640us) server_negotiation.cc:407] Connection header received
0928 16:02:40.476809 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.476815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.476902 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.477015 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.477758 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36386: BlockingRecv error: recv got EOF from 127.0.0.1:36386 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 16:02:40.490423 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.484761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.22.125.253:40687)
0928 16:02:40.485072 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.485080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.485567 (+ 487us) server_negotiation.cc:407] Connection header received
0928 16:02:40.485789 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.485796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.485861 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.485963 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.486408 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.487200 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.488720 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.489162 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.489300 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.489695 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.489797 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.489898 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.490045 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.490123 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
W20260928 16:02:40.508868 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.506625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.22.125.252:36983)
0928 16:02:40.506938 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.506948 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.507469 (+ 521us) server_negotiation.cc:407] Connection header received
0928 16:02:40.507677 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.507683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.507767 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.507878 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.508650 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36394: BlockingRecv error: recv got EOF from 127.0.0.1:36394 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
I20260928 16:02:40.528239 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.520393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.22.125.253:40687)
0928 16:02:40.520631 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.520653 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.521175 (+ 522us) server_negotiation.cc:407] Connection header received
0928 16:02:40.521401 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.521408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.521504 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.521606 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.522622 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.523786 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.525813 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.526525 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.526636 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.527272 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.527437 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.527504 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.527752 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.527848 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260928 16:02:40.536860 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.534704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.22.125.252:36983)
0928 16:02:40.534932 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.534940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.535471 (+ 531us) server_negotiation.cc:407] Connection header received
0928 16:02:40.535664 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.535670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.535755 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.535862 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.536625 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36402: BlockingRecv error: recv got EOF from 127.0.0.1:36402 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:02:40.562448 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.556500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.22.125.253:40687)
0928 16:02:40.556711 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.556724 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.557206 (+ 482us) server_negotiation.cc:407] Connection header received
0928 16:02:40.557430 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.557434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.557496 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.557593 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.557996 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.558869 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.560458 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.561113 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.561234 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.561672 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.561804 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.561902 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.562040 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.562120 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260928 16:02:40.570065 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.568553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36410 (local address 127.22.125.252:36983)
0928 16:02:40.568743 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.568749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.568767 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:02:40.568917 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.568923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.569010 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.569074 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.569882 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36410: BlockingRecv error: recv got EOF from 127.0.0.1:36410 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 16:02:40.617261 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.611750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.22.125.253:40687)
0928 16:02:40.611991 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.612000 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.612545 (+ 545us) server_negotiation.cc:407] Connection header received
0928 16:02:40.612765 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.612770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.612867 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.612968 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.613407 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.614148 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.615447 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.615897 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.616033 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.616471 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.616569 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.616674 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.616827 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.616903 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260928 16:02:40.632200 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.629760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.22.125.252:36983)
0928 16:02:40.629988 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.629997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.630796 (+ 799us) server_negotiation.cc:407] Connection header received
0928 16:02:40.631000 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.631010 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.631082 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.631197 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.631921 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36416: BlockingRecv error: recv got EOF from 127.0.0.1:36416 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 16:02:40.705466 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.699620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.22.125.253:40687)
0928 16:02:40.699846 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.699852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.700175 (+ 323us) server_negotiation.cc:407] Connection header received
0928 16:02:40.700379 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.700383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.700450 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.700521 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.700978 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.701837 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.703754 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.704242 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.704346 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.704788 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.704879 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.704944 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.705127 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.705193 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260928 16:02:40.717919 13333 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:40.711424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36432 (local address 127.22.125.252:36983)
0928 16:02:40.711660 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.711669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.716578 (+ 4909us) server_negotiation.cc:407] Connection header received
0928 16:02:40.716790 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.716795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.716872 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.716976 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:40.717711 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36432: BlockingRecv error: recv got EOF from 127.0.0.1:36432 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260928 16:02:40.723417 13291 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.22.125.252:36983
I20260928 16:02:40.807174 13312 catalog_manager.cc:1259] Loaded cluster ID: e67701f97d4f403da1106f632101ea27
I20260928 16:02:40.807315 13312 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: loading cluster ID for follower catalog manager: success
I20260928 16:02:40.809909 13312 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: acquiring CA information for follower catalog manager: success
I20260928 16:02:40.811710 13312 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 045c81fba9624184a9d919ca1c9a201e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:02:40.854992 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.849057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.22.125.253:40687)
0928 16:02:40.849362 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.849369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.849699 (+ 330us) server_negotiation.cc:407] Connection header received
0928 16:02:40.849938 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.849942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.850010 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.850123 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.850540 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.851297 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.853122 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.853706 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.853875 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.854281 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.854385 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.854484 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.854641 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.854723 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260928 16:02:40.866340 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.860441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.22.125.252:36983)
0928 16:02:40.860664 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.860670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.860902 (+ 232us) server_negotiation.cc:407] Connection header received
0928 16:02:40.861115 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.861121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.861307 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.861437 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.861916 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.862611 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.863948 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.864420 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.864553 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.865115 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.865684 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.865850 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.865928 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.866009 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:02:40.880769 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.871999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.22.125.254:35407)
0928 16:02:40.874432 (+ 2433us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.874439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.874462 (+ 23us) server_negotiation.cc:407] Connection header received
0928 16:02:40.874590 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.874597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.874832 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.875025 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.875585 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.876561 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.878319 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.878750 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.878889 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.879569 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.880011 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.880164 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.880286 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.880384 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2269,"thread_start_us":2152,"threads_started":1}
I20260928 16:02:40.881088 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.872544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.22.125.253:40687)
0928 16:02:40.872821 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.872829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.874809 (+ 1980us) server_negotiation.cc:407] Connection header received
0928 16:02:40.875095 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.875100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.875195 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.875246 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.876150 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.877056 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.879098 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.879678 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.879828 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.880313 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.880433 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.880499 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.880751 (+ 252us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.880851 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:02:40.882701 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:40.872274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36446 (local address 127.22.125.252:36983)
0928 16:02:40.872444 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:40.872452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:40.875117 (+ 2665us) server_negotiation.cc:407] Connection header received
0928 16:02:40.875385 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:40.875389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:40.875476 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:40.875560 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:40.876339 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.877468 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.879539 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:40.880189 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:40.880281 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:40.881887 (+ 1606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:40.882020 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:40.882086 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:40.882337 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 16:02:40.882425 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 16:02:40.885913 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 13078
I20260928 16:02:40.887034 13141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:40.996395 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 13078
W20260928 16:02:41.016901 13172 connection.cc:535] server connection from 127.0.0.1:42234 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 16:02:41.017455 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 13153
I20260928 16:02:41.018503 13215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:41.124692 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 13153
I20260928 16:02:41.144604 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 13227
I20260928 16:02:41.145726 13290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:41.252969 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 13227
2026-09-28T16:02:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4331 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 16:02:41 dist-test-slave-9cqd krb5kdc[13475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 16:02:41 dist-test-slave-9cqd krb5kdc[13475](info): set up 2 sockets
Sep 28 16:02:41 dist-test-slave-9cqd krb5kdc[13475](info): commencing operation
krb5kdc: starting...
W20260928 16:02:42.350489 23031 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/test-user.keytab.
Sep 28 16:02:42 dist-test-slave-9cqd krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T16:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T16:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.125.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
Entry for principal kudu/127.22.125.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab.
I20260928 16:02:42.615000 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:37403
--webserver_interface=127.22.125.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:37403,127.22.125.253:43175,127.22.125.252:41129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 16:02:42.938459 13488 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:02:42.938757 13488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:42.938822 13488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:42.938975 13488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:42.945938 13488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:42.946015 13488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:42.946167 13488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:42.946252 13488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:42.946341 13488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:42.955602 13488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:37403,127.22.125.253:43175,127.22.125.252:41129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.254.keytab
--principal=kudu/127.22.125.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.254:37403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.125.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:42.956781 13488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:42.958010 13488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:42.969681 13493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:42.969610 13496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:42.969656 13494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:42.971042 13488 server_base.cc:1034] running on GCE node
I20260928 16:02:42.971865 13488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:42.973256 13488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:42.974483 13488 hybrid_clock.cc:648] HybridClock initialized: now 1790611362974399 us; error 49 us; skew 500 ppm
I20260928 16:02:42.974822 13488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:42 dist-test-slave-9cqd krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:42.977916 13488 init.cc:377] Logged in from keytab as kudu/127.22.125.254@KRBTEST.COM (short username kudu)
I20260928 16:02:42.979343 13488 webserver.cc:460] Webserver started at http://127.22.125.254:42039/ using document root <none> and password file <none>
I20260928 16:02:42.980020 13488 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:42.980211 13488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:42.980530 13488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:42.983407 13488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data/instance:
uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4"
format_stamp: "Formatted at 2026-09-28 16:02:42 on dist-test-slave-9cqd"
server_key: "10e49db9780431aea6109135e6e88d9c"
server_key_iv: "fdec7e3c7032393b7de3fca4b603b8e6"
server_key_version: "encryptionkey@0"
I20260928 16:02:42.984267 13488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/wal/instance:
uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4"
format_stamp: "Formatted at 2026-09-28 16:02:42 on dist-test-slave-9cqd"
server_key: "10e49db9780431aea6109135e6e88d9c"
server_key_iv: "fdec7e3c7032393b7de3fca4b603b8e6"
server_key_version: "encryptionkey@0"
I20260928 16:02:42.989161 13488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 16:02:42.992461 13503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:42.993641 13488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260928 16:02:42.993827 13488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/wal
uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4"
format_stamp: "Formatted at 2026-09-28 16:02:42 on dist-test-slave-9cqd"
server_key: "10e49db9780431aea6109135e6e88d9c"
server_key_iv: "fdec7e3c7032393b7de3fca4b603b8e6"
server_key_version: "encryptionkey@0"
I20260928 16:02:42.994040 13488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:43.007463 13488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:43.011490 13488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:43.011780 13488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:43.030227 13488 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.254:37403
I20260928 16:02:43.030246 13554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.254:37403 every 8 connection(s)
I20260928 16:02:43.031478 13488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/data/info.pb
I20260928 16:02:43.034973 13555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:43.037240 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 13488
I20260928 16:02:43.037436 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-0/wal/instance
I20260928 16:02:43.037940 23031 external_mini_cluster.cc:1428] Setting key 3aceb793522e1b848c3abb1fccc2a7b6
I20260928 16:02:43.043896 13555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:43 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.253@KRBTEST.COM" created.
I20260928 16:02:43.067703 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.047610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33336 (local address 127.22.125.254:37403)
0928 16:02:43.049315 (+ 1705us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:43.049323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:43.049383 (+ 60us) server_negotiation.cc:407] Connection header received
0928 16:02:43.050819 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:43.050824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:43.051035 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:43.051252 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:43.052578 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.053772 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.055109 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.055636 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.055829 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.059577 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:43.059590 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:43.059608 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:43.059654 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:43.063528 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.064446 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.064452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.064454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.064598 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.064971 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.064974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.064976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.065509 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:43.065667 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:43.066289 (+ 622us) server_negotiation.cc:299] Negotiation successful
0928 16:02:43.066717 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":213,"server-negotiator.queue_time_us":1460,"thread_start_us":1210,"threads_started":1}
I20260928 16:02:43.067794 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.047605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37403 (local address 127.0.0.1:33336)
0928 16:02:43.048474 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.048519 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:43.050454 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:43.051262 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:43.051268 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:43.051969 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:43.052363 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.052381 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.053925 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.053931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.054967 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.054979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.055859 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.055865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.056093 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.056803 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:43.056828 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:43.059251 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:43.063762 (+ 4511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.063772 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.063779 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.064285 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.064694 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.064697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.064699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.064866 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.065637 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:43.065643 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:43.065851 (+ 208us) client_negotiation.cc:769] Sending connection context
0928 16:02:43.066135 (+ 284us) client_negotiation.cc:240] Negotiation successful
0928 16:02:43.066525 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":2,"thread_start_us":374,"threads_started":1}
I20260928 16:02:43.070751 13555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:43.071722 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.071365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38504)
0928 16:02:43.071517 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.071617 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 16:02:43.072079 13504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
I20260928 16:02:43.073529 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.073228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38518)
0928 16:02:43.073358 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.073421 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:02:43.073813 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:43175: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab.
Entry for principal kudu/127.22.125.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab.
I20260928 16:02:43.097532 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:43175
--webserver_interface=127.22.125.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab
--principal=kudu/127.22.125.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:37403,127.22.125.253:43175,127.22.125.252:41129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:02:43.123204 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } attempt: 1
I20260928 16:02:43.124487 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.123914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38526)
0928 16:02:43.124167 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.124307 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260928 16:02:43.125497 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.125119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38538)
0928 16:02:43.125268 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.125353 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 16:02:43.125681 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:43175: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
I20260928 16:02:43.193964 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } attempt: 2
I20260928 16:02:43.195355 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.194724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38542)
0928 16:02:43.195044 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.195175 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260928 16:02:43.196579 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.196163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38552)
0928 16:02:43.196330 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.196430 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260928 16:02:43.196768 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:43175: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
I20260928 16:02:43.288125 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } attempt: 3
I20260928 16:02:43.289532 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.288895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38560)
0928 16:02:43.289178 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.289319 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260928 16:02:43.290686 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.290260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38572)
0928 16:02:43.290440 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.290526 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260928 16:02:43.290905 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:43175: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
W20260928 16:02:43.425710 13563 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:02:43.426097 13563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:43.426230 13563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:43.426376 13563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 16:02:43.433760 13563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:43.433847 13563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:43.433944 13563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:43.434041 13563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:43.434156 13563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:43.434144 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } attempt: 4
I20260928 16:02:43.435398 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.434799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38576)
0928 16:02:43.435124 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.435233 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260928 16:02:43.436430 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.436012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38592)
0928 16:02:43.436194 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.436277 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260928 16:02:43.436638 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.253:43175: Network error: Client connection negotiation failed: client connection to 127.22.125.253:43175: connect: Connection refused (error 111)
I20260928 16:02:43.444900 13563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:37403,127.22.125.253:43175,127.22.125.252:41129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.253.keytab
--principal=kudu/127.22.125.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.253:43175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.125.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:43.446272 13563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:43.447613 13563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:43.459326 13568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:43.459398 13569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:43.459434 13571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:43.460275 13563 server_base.cc:1034] running on GCE node
I20260928 16:02:43.461035 13563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:43.462370 13563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:43.463635 13563 hybrid_clock.cc:648] HybridClock initialized: now 1790611363463551 us; error 57 us; skew 500 ppm
I20260928 16:02:43.463979 13563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:43 dist-test-slave-9cqd krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:43.467413 13563 init.cc:377] Logged in from keytab as kudu/127.22.125.253@KRBTEST.COM (short username kudu)
I20260928 16:02:43.469044 13563 webserver.cc:460] Webserver started at http://127.22.125.253:46109/ using document root <none> and password file <none>
I20260928 16:02:43.469759 13563 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:43.469894 13563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:43.470243 13563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:43.473408 13563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data/instance:
uuid: "533c34eddb0a482e8b35bb00f89b1117"
format_stamp: "Formatted at 2026-09-28 16:02:43 on dist-test-slave-9cqd"
server_key: "5615099de0ab81cd52a16e12959f2240"
server_key_iv: "0e898c63951d78f51891145daa83e94d"
server_key_version: "encryptionkey@0"
I20260928 16:02:43.474239 13563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/wal/instance:
uuid: "533c34eddb0a482e8b35bb00f89b1117"
format_stamp: "Formatted at 2026-09-28 16:02:43 on dist-test-slave-9cqd"
server_key: "5615099de0ab81cd52a16e12959f2240"
server_key_iv: "0e898c63951d78f51891145daa83e94d"
server_key_version: "encryptionkey@0"
I20260928 16:02:43.479537 13563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 16:02:43.483249 13578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:43.484861 13563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:43.485121 13563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/wal
uuid: "533c34eddb0a482e8b35bb00f89b1117"
format_stamp: "Formatted at 2026-09-28 16:02:43 on dist-test-slave-9cqd"
server_key: "5615099de0ab81cd52a16e12959f2240"
server_key_iv: "0e898c63951d78f51891145daa83e94d"
server_key_version: "encryptionkey@0"
I20260928 16:02:43.485371 13563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:43.503240 13563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:43.507826 13563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:43.508183 13563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:43.527036 13563 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.253:43175
I20260928 16:02:43.527058 13629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.253:43175 every 8 connection(s)
I20260928 16:02:43.528463 13563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/data/info.pb
I20260928 16:02:43.531351 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 13563
I20260928 16:02:43.531556 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-1/wal/instance
I20260928 16:02:43.532007 23031 external_mini_cluster.cc:1428] Setting key 7c3f23b7ca81abe7788b4438bfb5086a
I20260928 16:02:43.532231 13630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:43.538514 13630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:43 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.125.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.125.252@KRBTEST.COM" created.
I20260928 16:02:43.559849 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.541208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.22.125.254:37403)
0928 16:02:43.541446 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:43.541455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:43.542544 (+ 1089us) server_negotiation.cc:407] Connection header received
0928 16:02:43.544494 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:43.544503 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:43.544606 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:43.544728 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:43.546190 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.547326 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.548937 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.549723 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.549870 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.553283 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:43.553290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:43.553296 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:43.553343 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:43.556654 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.557392 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.557397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.557399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.557527 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.557891 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.557894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.557896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.558170 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:43.558284 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:43.559150 (+ 866us) server_negotiation.cc:299] Negotiation successful
0928 16:02:43.559430 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 16:02:43.560549 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.541572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37403 (local address 127.0.0.1:33350)
0928 16:02:43.542363 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.542414 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:43.544224 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:43.544912 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:43.544919 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:43.545451 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:43.545995 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.546012 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.547490 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.547494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.548792 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.548805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.549930 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.549935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.550121 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.550840 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:43.550862 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:43.553046 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:43.556789 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.556795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.556809 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.557254 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.557625 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.557628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.557630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.557791 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.558289 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:43.558295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:43.558741 (+ 446us) client_negotiation.cc:769] Sending connection context
0928 16:02:43.558971 (+ 230us) client_negotiation.cc:240] Negotiation successful
0928 16:02:43.559346 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":342,"threads_started":1}
I20260928 16:02:43.562355 13630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:43 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:02:43.577173 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.563266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38596 (local address 127.22.125.253:43175)
0928 16:02:43.563892 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:43.563897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:43.563922 (+ 25us) server_negotiation.cc:407] Connection header received
0928 16:02:43.563965 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:43.563970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:43.564197 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:43.564364 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:43.565093 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.566145 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.567248 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.567773 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.567908 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.571415 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:43.571425 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:43.571438 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:43.571465 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:43.574237 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.575077 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.575083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.575086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.575196 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.575611 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.575613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.575615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.575945 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:43.576154 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:43.576567 (+ 413us) server_negotiation.cc:299] Negotiation successful
0928 16:02:43.576882 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":157,"server-negotiator.queue_time_us":491,"thread_start_us":344,"threads_started":1}
I20260928 16:02:43.577284 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.562998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38596)
0928 16:02:43.563217 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.563232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:43.563310 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:43.564354 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:43.564358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:43.564796 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:43.564920 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.564928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.566276 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.566281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.567113 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.567124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.567915 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.567920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.568160 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.568740 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:43.568757 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:43.571183 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:43.574386 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.574394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.574401 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.574951 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.575309 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.575316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.575322 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.575506 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.576108 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:43.576115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:43.576281 (+ 166us) client_negotiation.cc:769] Sending connection context
0928 16:02:43.576438 (+ 157us) client_negotiation.cc:240] Negotiation successful
0928 16:02:43.576765 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":3}
I20260928 16:02:43.579792 13630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:43.581067 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.580465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52736)
0928 16:02:43.580689 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.580853 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260928 16:02:43.581379 13581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
I20260928 16:02:43.583052 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.582571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52748)
0928 16:02:43.582770 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.582892 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260928 16:02:43.583309 13630 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.125.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab.
Entry for principal kudu/127.22.125.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab.
I20260928 16:02:43.592485 23031 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
/tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:41129
--webserver_interface=127.22.125.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab
--principal=kudu/127.22.125.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.125.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.125.254:37403,127.22.125.253:43175,127.22.125.252:41129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 16:02:43.632627 13630 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 1
I20260928 16:02:43.633872 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.633273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52760)
0928 16:02:43.633536 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.633672 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 16:02:43.634953 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.634541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52762)
0928 16:02:43.634718 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.634820 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:02:43.635162 13630 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
I20260928 16:02:43.703408 13630 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 2
I20260928 16:02:43.704689 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.704159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52774)
0928 16:02:43.704379 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.704496 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 16:02:43.705746 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.705350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52782)
0928 16:02:43.705507 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.705596 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:02:43.705960 13630 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
I20260928 16:02:43.735926 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } attempt: 5
Sep 28 16:02:43 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:02:43.749768 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.736797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38604)
0928 16:02:43.737002 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.737030 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:43.737201 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:43.737671 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:43.737676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:43.738088 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:43.738250 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.738260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.739261 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.739269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.740208 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:43.740220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.740985 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.740991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:43.741213 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.741865 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:43.741883 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:43.744291 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:43.747303 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.747309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.747313 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.747689 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.748098 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:43.748103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:43.748107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:43.748258 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:43.748782 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:43.748787 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:43.748942 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 16:02:43.749078 (+ 136us) client_negotiation.cc:240] Negotiation successful
0928 16:02:43.749411 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 16:02:43.749768 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.737063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.22.125.253:43175)
0928 16:02:43.737318 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:43.737324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:43.737342 (+ 18us) server_negotiation.cc:407] Connection header received
0928 16:02:43.737421 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:43.737425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:43.737488 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:43.737602 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:43.738410 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.739121 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.740371 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:43.740822 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:43.740976 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:43.744486 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:43.744490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:43.744494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:43.744524 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:43.747155 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.747819 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.747831 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.747834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.747956 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:43.748367 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:43.748374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:43.748377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:43.748661 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:43.748818 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:43.749220 (+ 402us) server_negotiation.cc:299] Negotiation successful
0928 16:02:43.749445 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 16:02:43.751376 13555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:43.752522 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.752032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52796)
0928 16:02:43.752262 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.752357 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 16:02:43.753647 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.753189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52808)
0928 16:02:43.753368 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.753487 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 16:02:43.753892 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
I20260928 16:02:43.797196 13630 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 3
I20260928 16:02:43.798398 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.797850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52820)
0928 16:02:43.798075 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.798202 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260928 16:02:43.799535 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.799081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52826)
0928 16:02:43.799311 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.799407 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260928 16:02:43.799763 13630 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
I20260928 16:02:43.805154 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 1
I20260928 16:02:43.806313 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.805731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52840)
0928 16:02:43.805956 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.806102 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 16:02:43.807461 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.807005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52842)
0928 16:02:43.807189 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.807292 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 16:02:43.807758 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
I20260928 16:02:43.876014 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 2
I20260928 16:02:43.877344 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.876720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52848)
0928 16:02:43.876996 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.877118 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260928 16:02:43.878491 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.878047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52850)
0928 16:02:43.878237 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.878342 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 16:02:43.878748 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
W20260928 16:02:43.936806 13637 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 16:02:43.937212 13637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 16:02:43.937326 13637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 16:02:43.937462 13637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260928 16:02:43.942981 13630 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 4
I20260928 16:02:43.944168 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.943615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52864)
0928 16:02:43.943852 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.943949 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260928 16:02:43.945063 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.944783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52872)
0928 16:02:43.944905 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.944969 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 16:02:43.945175 13637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 16:02:43.945290 13637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 16:02:43.945328 13630 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
W20260928 16:02:43.945377 13637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 16:02:43.945457 13637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 16:02:43.945544 13637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 16:02:43.955083 13637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.125.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.125.254:37403,127.22.125.253:43175,127.22.125.252:41129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/krb5kdc/kudu_127.22.125.252.keytab
--principal=kudu/127.22.125.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.125.252:41129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.125.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:11:15 UTC on ccycloud.quasar-ubnjdo.root.comops.site
build id 4677204
ASAN enabled
I20260928 16:02:43.956344 13637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 16:02:43.957659 13637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 16:02:43.968906 13642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:43.968997 13645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 16:02:43.969225 13643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 16:02:43.970264 13637 server_base.cc:1034] running on GCE node
I20260928 16:02:43.970983 13637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 16:02:43.972450 13637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 16:02:43.973708 13637 hybrid_clock.cc:648] HybridClock initialized: now 1790611363973661 us; error 38 us; skew 500 ppm
I20260928 16:02:43.974030 13637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 16:02:43 dist-test-slave-9cqd krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 16:02:43.977072 13637 init.cc:377] Logged in from keytab as kudu/127.22.125.252@KRBTEST.COM (short username kudu)
I20260928 16:02:43.978454 13637 webserver.cc:460] Webserver started at http://127.22.125.252:42567/ using document root <none> and password file <none>
I20260928 16:02:43.979122 13637 fs_manager.cc:359] Metadata directory not provided
I20260928 16:02:43.979256 13637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 16:02:43.979566 13637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 16:02:43.982475 13637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data/instance:
uuid: "fc780d0aca6b406898f0f503c7c18801"
format_stamp: "Formatted at 2026-09-28 16:02:43 on dist-test-slave-9cqd"
server_key: "c9c2650039344c3ad922651e311402bf"
server_key_iv: "fa27c98a5aded302492dc62550427e68"
server_key_version: "encryptionkey@0"
I20260928 16:02:43.983307 13637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/wal/instance:
uuid: "fc780d0aca6b406898f0f503c7c18801"
format_stamp: "Formatted at 2026-09-28 16:02:43 on dist-test-slave-9cqd"
server_key: "c9c2650039344c3ad922651e311402bf"
server_key_iv: "fa27c98a5aded302492dc62550427e68"
server_key_version: "encryptionkey@0"
I20260928 16:02:43.984993 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 3
I20260928 16:02:43.986029 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.985571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52886)
0928 16:02:43.985798 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.985879 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260928 16:02:43.987001 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:43.986644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52902)
0928 16:02:43.986841 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:43.986903 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260928 16:02:43.987316 13555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.125.252:41129: Network error: Client connection negotiation failed: client connection to 127.22.125.252:41129: connect: Connection refused (error 111)
I20260928 16:02:43.988509 13637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 16:02:43.991663 13652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 16:02:43.993124 13637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 16:02:43.993335 13637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data,/tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/wal
uuid: "fc780d0aca6b406898f0f503c7c18801"
format_stamp: "Formatted at 2026-09-28 16:02:43 on dist-test-slave-9cqd"
server_key: "c9c2650039344c3ad922651e311402bf"
server_key_iv: "fa27c98a5aded302492dc62550427e68"
server_key_version: "encryptionkey@0"
I20260928 16:02:43.993557 13637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 16:02:44.016093 13637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 16:02:44.020514 13637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 16:02:44.020799 13637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 16:02:44.039830 13637 rpc_server.cc:307] RPC server started. Bound to: 127.22.125.252:41129
I20260928 16:02:44.039875 13703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.125.252:41129 every 8 connection(s)
I20260928 16:02:44.041122 13637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/data/info.pb
I20260928 16:02:44.044711 13704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 16:02:44.049067 23031 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu as pid 13637
I20260928 16:02:44.049265 23031 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYsP_Ep/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790611005938867-23031-0/minicluster-data/master-2/wal/instance
I20260928 16:02:44.049814 23031 external_mini_cluster.cc:1428] Setting key e3e84f2a131e6610f3084f341b3e2895
I20260928 16:02:44.052024 13704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:02:44.070839 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.055850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.22.125.254:37403)
0928 16:02:44.056082 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.056088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.056789 (+ 701us) server_negotiation.cc:407] Connection header received
0928 16:02:44.056979 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.056985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.057073 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.057209 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.058326 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.059000 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.060449 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.060977 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.061116 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.065448 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.065452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.065456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.065486 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.067971 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.068699 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.068703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.068705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.068839 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.069248 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.069251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.069253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.069509 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.069619 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.070278 (+ 659us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.070502 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 16:02:44.072175 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.055790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.254:37403 (local address 127.0.0.1:33362)
0928 16:02:44.056628 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:44.056674 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:44.056843 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:44.057259 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:44.057264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:44.057882 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:44.058152 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.058165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.059149 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.059154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.060309 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.060321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.061209 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.061217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.061452 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.061997 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:44.062023 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:44.065244 (+ 3221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:44.068112 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.068125 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.068137 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.068550 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.068973 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.068976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.068978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.069112 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.069634 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:44.069641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:44.069894 (+ 253us) client_negotiation.cc:769] Sending connection context
0928 16:02:44.070160 (+ 266us) client_negotiation.cc:240] Negotiation successful
0928 16:02:44.070731 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":133,"thread_start_us":467,"threads_started":1}
I20260928 16:02:44.072194 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.051568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52914 (local address 127.22.125.252:41129)
0928 16:02:44.052385 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.052396 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.052435 (+ 39us) server_negotiation.cc:407] Connection header received
0928 16:02:44.053955 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.053977 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.054495 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.054818 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.056471 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.058138 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.059383 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.060151 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.060345 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.064982 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.064992 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.065005 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.065050 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.067991 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.068647 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.068654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.068658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.068809 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.069200 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.069205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.069209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.069745 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.070019 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.070253 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.070742 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":597,"thread_start_us":424,"threads_started":1}
I20260928 16:02:44.073783 13704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.254@KRBTEST.COM
I20260928 16:02:44.087658 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.074384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.253:43175 (local address 127.0.0.1:38614)
0928 16:02:44.074778 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:44.074795 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:44.074901 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:44.075593 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:44.075598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:44.075960 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:44.076157 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.076166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.077097 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.077105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.077965 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.077976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.078653 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.078659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.078870 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.079413 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:44.079428 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:44.081476 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:44.085338 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.085362 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.085366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.085722 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.086134 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.086136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.086139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.086260 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.086854 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:44.086857 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:44.086973 (+ 116us) client_negotiation.cc:769] Sending connection context
0928 16:02:44.087083 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 16:02:44.087264 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289}
I20260928 16:02:44.087682 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.074495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.22.125.253:43175)
0928 16:02:44.074682 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.074688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.074867 (+ 179us) server_negotiation.cc:407] Connection header received
0928 16:02:44.075396 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.075400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.075463 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.075575 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.076308 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.076964 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.078103 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.078507 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.078634 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.081665 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.081669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.081673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.081702 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.085161 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.085842 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.085850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.085855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.086010 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.086376 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.086383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.086387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.086740 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.086893 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.087204 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.087431 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 16:02:44.088660 13704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } has no permanent_uuid. Determining permanent_uuid...
I20260928 16:02:44.091130 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.075247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33366 (local address 127.22.125.254:37403)
0928 16:02:44.075405 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.075410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.075453 (+ 43us) server_negotiation.cc:407] Connection header received
0928 16:02:44.075628 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.075634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.075691 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.075804 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.076949 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.077684 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.078878 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.079270 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.079396 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.083474 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.083478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.083481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.083540 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.087214 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.087985 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.087990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.087994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.088243 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.088688 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.088692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.088696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.089028 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.089270 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.090750 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.090911 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.252@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.125.253@KRBTEST.COM
I20260928 16:02:44.102465 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.089438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52920)
0928 16:02:44.089608 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:44.089622 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:44.089692 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:44.090112 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:44.090115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:44.090490 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:44.090612 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.090621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.091694 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.091702 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.092812 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.092824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.093458 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.093463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.093713 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.094300 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:44.094316 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:44.096140 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:44.099983 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.099986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.099989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.100390 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.100751 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.100757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.100760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.100929 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.101423 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:44.101427 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:44.101563 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 16:02:44.101723 (+ 160us) client_negotiation.cc:240] Negotiation successful
0928 16:02:44.102022 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":16}
I20260928 16:02:44.102484 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.089695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.22.125.252:41129)
0928 16:02:44.089847 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.089852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.089866 (+ 14us) server_negotiation.cc:407] Connection header received
0928 16:02:44.089911 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.089915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.089984 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.090081 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.090745 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.091577 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.092939 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.093325 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.093451 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.096327 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.096332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.096336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.096361 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.099867 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.100501 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.100509 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.100512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.100635 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.101040 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.101044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.101067 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.101313 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.101458 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.101846 (+ 388us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.102160 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":55}
I20260928 16:02:44.107050 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.092698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.22.125.253:43175)
0928 16:02:44.092869 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.092875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.092891 (+ 16us) server_negotiation.cc:407] Connection header received
0928 16:02:44.092992 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.092997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.093057 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.093155 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.094271 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.094982 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.096314 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.096723 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.096857 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.101688 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.101695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.101702 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.101739 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.104513 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.105157 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.105169 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.105174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.105344 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.105710 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.105717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.105720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.105957 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.106271 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.106523 (+ 252us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.106743 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:02:44.107513 13704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: Bootstrap starting.
I20260928 16:02:44.111299 13704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:44.111603 23031 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 16:02:44.112432 13704 log.cc:824] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:44.115622 13704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: No bootstrap required, opened a new log
I20260928 16:02:44.114215 13530 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33366
I20260928 16:02:44.126260 13704 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.126645 13704 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc780d0aca6b406898f0f503c7c18801, State: Initialized, Role: FOLLOWER
I20260928 16:02:44.127144 13704 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.128010 13711 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.128407 13711 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:44.129225 13704 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 16:02:44.138259 13722 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 16:02:44.138381 13722 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 16:02:44.135946 13637 master_runner.cc:186] Error getting master registration for 127.22.125.254:37403: OK, instance_id { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" instance_seqno: 1790611363025095 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:02:44.146867 13637 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:02:44.164539 13555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 4
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611362, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.254@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
I20260928 16:02:44.179203 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.165498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52930 (local address 127.22.125.252:41129)
0928 16:02:44.165665 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.165670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.165692 (+ 22us) server_negotiation.cc:407] Connection header received
0928 16:02:44.165814 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.165820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.165889 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.166002 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.166777 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.167544 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.168845 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.169329 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.169472 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.173508 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.173515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.173519 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.173564 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.176550 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.177190 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.177197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.177200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.177320 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.177694 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.177700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.177704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.178061 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.178230 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.178628 (+ 398us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.178848 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 16:02:44.179189 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.165327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52930)
0928 16:02:44.165518 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:44.165539 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:44.165670 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:44.166030 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:44.166034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:44.166457 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:44.166627 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.166635 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.167694 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.167699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.168701 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.168713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.169492 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.169497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.169701 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.170377 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:44.170399 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:44.173310 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:44.176688 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.176695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.176699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.177069 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.177429 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.177434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.177438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.177584 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.178199 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:44.178204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:44.178352 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 16:02:44.178496 (+ 144us) client_negotiation.cc:240] Negotiation successful
0928 16:02:44.178775 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 16:02:44.185617 13555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4: Bootstrap starting.
I20260928 16:02:44.189797 13555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:44.191103 13555 log.cc:824] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:44.194670 13555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4: No bootstrap required, opened a new log
I20260928 16:02:44.202451 13555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.202849 13555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857e95ed03fe4d5fa3a5f43bb706ccc4, State: Initialized, Role: FOLLOWER
I20260928 16:02:44.203374 13555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.204370 13725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.204806 13725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:44.205647 13555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:44.205714 13530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 16:02:44.205963 13530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:44.208721 13530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.211151 13679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fc780d0aca6b406898f0f503c7c18801"
I20260928 16:02:44.211648 13530 leader_election.cc:288] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [CANDIDATE]: Term 1 election: Requested vote from peers 533c34eddb0a482e8b35bb00f89b1117 (127.22.125.253:43175), fc780d0aca6b406898f0f503c7c18801 (127.22.125.252:41129)
I20260928 16:02:44.211709 13679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:44.215889 13679 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 857e95ed03fe4d5fa3a5f43bb706ccc4 in term 1.
I20260928 16:02:44.216687 13506 leader_election.cc:302] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857e95ed03fe4d5fa3a5f43bb706ccc4, fc780d0aca6b406898f0f503c7c18801; no voters:
I20260928 16:02:44.217247 13725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 16:02:44.221761 13725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [term 1 LEADER]: Becoming Leader. State: Replica: 857e95ed03fe4d5fa3a5f43bb706ccc4, State: Running, Role: LEADER
I20260928 16:02:44.222471 13725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.237445 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 857e95ed03fe4d5fa3a5f43bb706ccc4. Latest consensus state: current_term: 1 leader_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.237890 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:44.238897 13738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 16:02:44.244535 13630 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } attempt: 5
Sep 28 16:02:44 dist-test-slave-9cqd krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790611363, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.125.253@KRBTEST.COM for kudu/127.22.125.252@KRBTEST.COM
I20260928 16:02:44.259554 13738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260928 16:02:44.264562 13509 tablet.cc:2369] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:02:44.270361 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.246291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.125.252:41129 (local address 127.0.0.1:52934)
0928 16:02:44.246565 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0928 16:02:44.246585 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 16:02:44.246715 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 16:02:44.247368 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 16:02:44.247374 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 16:02:44.247873 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0928 16:02:44.248102 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.248115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.249633 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.249639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.251059 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 16:02:44.251074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.252377 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.252383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 16:02:44.252626 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.253413 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 16:02:44.253460 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0928 16:02:44.257497 (+ 4037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 16:02:44.263062 (+ 5565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.263068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.263072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.263904 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.265713 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 16:02:44.265720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 16:02:44.265725 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 16:02:44.266123 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 16:02:44.267231 (+ 1108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 16:02:44.267237 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 16:02:44.267506 (+ 269us) client_negotiation.cc:769] Sending connection context
0928 16:02:44.267761 (+ 255us) client_negotiation.cc:240] Negotiation successful
0928 16:02:44.268932 (+ 1171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260928 16:02:44.270561 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.245585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.22.125.252:41129)
0928 16:02:44.246016 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.246078 (+ 62us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.246842 (+ 764us) server_negotiation.cc:407] Connection header received
0928 16:02:44.246923 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.246928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.247049 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.247142 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.248305 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.249433 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.251272 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.252003 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.252148 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.257766 (+ 5618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.257773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.257777 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.257819 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.262850 (+ 5031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.264104 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.264113 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.264151 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.265508 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.266325 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.266333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.266337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.267039 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.267983 (+ 944us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.268458 (+ 475us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.269254 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
W20260928 16:02:44.273299 13658 tablet.cc:2369] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 16:02:44.280411 13630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117: Bootstrap starting.
I20260928 16:02:44.287930 13630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117: Neither blocks nor log segments found. Creating new log.
I20260928 16:02:44.289768 13630 log.cc:824] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117: Log is configured to *not* fsync() on all Append() calls
I20260928 16:02:44.302434 13679 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [term 1 FOLLOWER]: Refusing update from remote peer 857e95ed03fe4d5fa3a5f43bb706ccc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:02:44.303630 13734 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:02:44.284448 13488 master_runner.cc:186] Error getting master registration for 127.22.125.253:43175: OK, instance_id { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" instance_seqno: 1790611363521861 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 16:02:44.314847 13488 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 16:02:44.330796 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.290453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.22.125.254:37403)
0928 16:02:44.290685 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.290693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.302349 (+ 11656us) server_negotiation.cc:407] Connection header received
0928 16:02:44.303710 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.303719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.303821 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.303899 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.308275 (+ 4376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.309789 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.313612 (+ 3823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.314338 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.314442 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.323436 (+ 8994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.323443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.323448 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.323498 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.327215 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.328230 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.328235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.328238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.328404 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.328894 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.328902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.328906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.329291 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.329534 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.330133 (+ 599us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.330393 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 16:02:44.332588 13711 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 857e95ed03fe4d5fa3a5f43bb706ccc4. Latest consensus state: current_term: 1 leader_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.332937 13711 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:44.335527 13744 mvcc.cc:205] Tried to move back new op lower bound from 7334344148160000000 to 7334344147862130688. Current Snapshot: MvccSnapshot[applied={T|T < 7334344148160000000}]
I20260928 16:02:44.343451 13750 mvcc.cc:205] Tried to move back new op lower bound from 7334344148160000000 to 7334344147862130688. Current Snapshot: MvccSnapshot[applied={T|T < 7334344148160000000}]
I20260928 16:02:44.345103 13630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117: No bootstrap required, opened a new log
I20260928 16:02:44.346679 13738 catalog_manager.cc:1347] Generated new cluster ID: a74f795d5a0940ce9992e6dc364616f8
I20260928 16:02:44.346781 13738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 16:02:44.348341 13725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.348650 13725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: This master's current role is: LEADER
I20260928 16:02:44.354614 13711 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.354882 13711 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:44.358373 13630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.358790 13630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 533c34eddb0a482e8b35bb00f89b1117, State: Initialized, Role: FOLLOWER
I20260928 16:02:44.359279 13630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } }
I20260928 16:02:44.360176 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.289939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.22.125.252:41129)
0928 16:02:44.290152 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.290161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.303252 (+ 13091us) server_negotiation.cc:407] Connection header received
0928 16:02:44.304003 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.304009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.304124 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.304191 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.320055 (+ 15864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.321196 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.333081 (+ 11885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.333789 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.333903 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.350637 (+ 16734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.350646 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.350653 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.350700 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.354770 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.357660 (+ 2890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.357665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.357668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.357838 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.358296 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.358301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.358304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.358669 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.358944 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.359536 (+ 592us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.359775 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 16:02:44.372686 13738 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 16:02:44.375088 13738 catalog_manager.cc:1504] Loading token signing keys...
I20260928 16:02:44.379940 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.302992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.22.125.253:43175)
0928 16:02:44.303215 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.303221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.326854 (+ 23633us) server_negotiation.cc:407] Connection header received
0928 16:02:44.327898 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.327905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.327991 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.328131 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.344167 (+ 16036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.345505 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.348796 (+ 3291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.349589 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.349708 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.372241 (+ 22533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 16:02:44.372250 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 16:02:44.372258 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 16:02:44.372319 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0928 16:02:44.376343 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.377314 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.377321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.377324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.377503 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 16:02:44.377978 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 16:02:44.377983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 16:02:44.377986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 16:02:44.378402 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 16:02:44.378648 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.379305 (+ 657us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.379565 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260928 16:02:44.384558 13754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.385007 13754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:44.387910 13630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 16:02:44.387974 13605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "533c34eddb0a482e8b35bb00f89b1117"
I20260928 16:02:44.388484 13605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [term 0 FOLLOWER]: Advancing to term 1
I20260928 16:02:44.392073 13605 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 857e95ed03fe4d5fa3a5f43bb706ccc4 in term 1.
I20260928 16:02:44.393065 13604 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [term 1 FOLLOWER]: Refusing update from remote peer 857e95ed03fe4d5fa3a5f43bb706ccc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 16:02:44.394265 13734 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 16:02:44.395121 13738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4: Generated new TSK 0
I20260928 16:02:44.396533 13738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 16:02:44.446368 13759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.446813 13759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:44.447158 13754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 857e95ed03fe4d5fa3a5f43bb706ccc4. Latest consensus state: current_term: 1 leader_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.447404 13754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 16:02:44.532998 13773 catalog_manager.cc:1259] Loaded cluster ID: a74f795d5a0940ce9992e6dc364616f8
I20260928 16:02:44.533186 13773 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117: loading cluster ID for follower catalog manager: success
I20260928 16:02:44.537689 13773 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117: acquiring CA information for follower catalog manager: success
I20260928 16:02:44.540225 13773 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 533c34eddb0a482e8b35bb00f89b1117: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 16:02:44.577566 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.562168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.22.125.252:41129)
0928 16:02:44.562541 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.562548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.572878 (+ 10330us) server_negotiation.cc:407] Connection header received
0928 16:02:44.575467 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.575474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.575561 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.575637 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.577302 (+ 1665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52948: BlockingRecv error: recv got EOF from 127.0.0.1:52948 (error 108)
Metrics: {"server-negotiator.queue_time_us":266}
W20260928 16:02:44.579300 13559 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.562360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.22.125.254:37403)
0928 16:02:44.562528 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.562536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.573384 (+ 10848us) server_negotiation.cc:407] Connection header received
0928 16:02:44.575029 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.575036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.575148 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.575232 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.579022 (+ 3790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33382: BlockingRecv error: recv got EOF from 127.0.0.1:33382 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 16:02:44.582929 13634 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.573958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.22.125.253:43175)
0928 16:02:44.574178 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.574185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.580550 (+ 6365us) server_negotiation.cc:407] Connection header received
0928 16:02:44.581630 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.581635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.581719 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.581792 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.582687 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38650: BlockingRecv error: recv got EOF from 127.0.0.1:38650 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 16:02:44.633157 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.612393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.22.125.253:43175)
0928 16:02:44.612618 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.612628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.624606 (+ 11978us) server_negotiation.cc:407] Connection header received
0928 16:02:44.624879 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.624886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.625091 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.625250 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:44.625927 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.627065 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.629605 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.630332 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.630462 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.631363 (+ 901us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:44.632165 (+ 802us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:44.632299 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.632596 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.632740 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260928 16:02:44.690886 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.672231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52960 (local address 127.22.125.252:41129)
0928 16:02:44.672454 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.672461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.688731 (+ 16270us) server_negotiation.cc:407] Connection header received
0928 16:02:44.689099 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.689105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.689259 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.689395 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.690618 (+ 1223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52960: BlockingRecv error: recv got EOF from 127.0.0.1:52960 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 16:02:44.744904 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.725979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38666 (local address 127.22.125.253:43175)
0928 16:02:44.726260 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.726268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.737070 (+ 10802us) server_negotiation.cc:407] Connection header received
0928 16:02:44.737682 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.737690 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.737800 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.737877 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:44.738627 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.739953 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.742296 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.743084 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.743208 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.743894 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:44.744101 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:44.744183 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.744434 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.744560 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260928 16:02:44.789228 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.779079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52974 (local address 127.22.125.252:41129)
0928 16:02:44.779300 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.779307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.786795 (+ 7488us) server_negotiation.cc:407] Connection header received
0928 16:02:44.787326 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.787334 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.787450 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.787536 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.788856 (+ 1320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52974: BlockingRecv error: recv got EOF from 127.0.0.1:52974 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260928 16:02:44.817874 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.803873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.22.125.253:43175)
0928 16:02:44.804117 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.804124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.810126 (+ 6002us) server_negotiation.cc:407] Connection header received
0928 16:02:44.810366 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.810374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.810481 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.810596 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:44.811437 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.812841 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.815231 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.815964 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.816117 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.816799 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:44.817005 (+ 206us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:44.817076 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.817418 (+ 342us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.817516 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260928 16:02:44.839174 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.836730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52978 (local address 127.22.125.252:41129)
0928 16:02:44.836923 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.836930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.837501 (+ 571us) server_negotiation.cc:407] Connection header received
0928 16:02:44.837726 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.837733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.837843 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.837971 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.838910 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52978: BlockingRecv error: recv got EOF from 127.0.0.1:52978 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 16:02:44.854563 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.846835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.22.125.253:43175)
0928 16:02:44.847077 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.847084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.847664 (+ 580us) server_negotiation.cc:407] Connection header received
0928 16:02:44.847877 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.847883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.847992 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.848147 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:44.848679 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.849975 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.852206 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.852899 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.853036 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.853667 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:44.853807 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:44.853912 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.854106 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.854244 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260928 16:02:44.870258 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.862215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.22.125.252:41129)
0928 16:02:44.862482 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.862488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.868563 (+ 6075us) server_negotiation.cc:407] Connection header received
0928 16:02:44.868882 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.868888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.868998 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.869127 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.869983 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52980: BlockingRecv error: recv got EOF from 127.0.0.1:52980 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
I20260928 16:02:44.889022 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.881581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.22.125.253:43175)
0928 16:02:44.881774 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.881780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.882283 (+ 503us) server_negotiation.cc:407] Connection header received
0928 16:02:44.882471 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.882477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.882582 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.882685 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:44.883368 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.884624 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.886683 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.887385 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.887510 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.888028 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:44.888216 (+ 188us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:44.888303 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.888505 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.888699 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260928 16:02:44.900583 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.898190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.22.125.252:41129)
0928 16:02:44.898366 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.898372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.898941 (+ 569us) server_negotiation.cc:407] Connection header received
0928 16:02:44.899154 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.899160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.899253 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.899324 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.900379 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52990: BlockingRecv error: recv got EOF from 127.0.0.1:52990 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 16:02:44.928864 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.921603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.22.125.253:43175)
0928 16:02:44.921819 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.921827 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.922371 (+ 544us) server_negotiation.cc:407] Connection header received
0928 16:02:44.922564 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.922570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.922667 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.922769 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:44.923258 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.924491 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.926572 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.927268 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.927387 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.927985 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:44.928180 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:44.928273 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.928435 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.928538 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260928 16:02:44.943843 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:44.941951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52992 (local address 127.22.125.252:41129)
0928 16:02:44.942153 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.942160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.942466 (+ 306us) server_negotiation.cc:407] Connection header received
0928 16:02:44.942618 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.942625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.942726 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.942840 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:44.943602 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52992: BlockingRecv error: recv got EOF from 127.0.0.1:52992 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 16:02:44.985338 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:44.979178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.22.125.253:43175)
0928 16:02:44.979407 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:44.979414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:44.979787 (+ 373us) server_negotiation.cc:407] Connection header received
0928 16:02:44.980007 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:44.980014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:44.980129 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:44.980255 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:44.980646 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.981867 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.983200 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:44.983937 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:44.984099 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:44.984521 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:44.984665 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:44.984763 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:44.984904 (+ 141us) server_negotiation.cc:299] Negotiation successful
0928 16:02:44.984987 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:02:44.987102 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857e95ed03fe4d5fa3a5f43bb706ccc4" member_type: VOTER last_known_addr { host: "127.22.125.254" port: 37403 } } peers { permanent_uuid: "533c34eddb0a482e8b35bb00f89b1117" member_type: VOTER last_known_addr { host: "127.22.125.253" port: 43175 } } peers { permanent_uuid: "fc780d0aca6b406898f0f503c7c18801" member_type: VOTER last_known_addr { host: "127.22.125.252" port: 41129 } } }
I20260928 16:02:44.987392 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857e95ed03fe4d5fa3a5f43bb706ccc4 [sys.catalog]: This master's current role is: LEADER
W20260928 16:02:45.002130 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:45.000158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.22.125.252:41129)
0928 16:02:45.000358 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.000363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.000639 (+ 276us) server_negotiation.cc:407] Connection header received
0928 16:02:45.000829 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.000834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.000907 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.000989 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:45.001891 (+ 902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53008: BlockingRecv error: recv got EOF from 127.0.0.1:53008 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260928 16:02:45.076332 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:45.070268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.22.125.253:43175)
0928 16:02:45.070486 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.070494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.070906 (+ 412us) server_negotiation.cc:407] Connection header received
0928 16:02:45.071109 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.071116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.071211 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.071325 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:45.071716 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.072982 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.074337 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.075014 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.075146 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:45.075553 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:45.075658 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:45.075768 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:45.075940 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 16:02:45.076023 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260928 16:02:45.084754 13707 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 16:02:45.082669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.22.125.252:41129)
0928 16:02:45.082913 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.082918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.083236 (+ 318us) server_negotiation.cc:407] Connection header received
0928 16:02:45.083423 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.083428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.083512 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.083592 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 16:02:45.084549 (+ 957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53012: BlockingRecv error: recv got EOF from 127.0.0.1:53012 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260928 16:02:45.140578 13722 catalog_manager.cc:1259] Loaded cluster ID: a74f795d5a0940ce9992e6dc364616f8
I20260928 16:02:45.140676 13722 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: loading cluster ID for follower catalog manager: success
I20260928 16:02:45.143150 13722 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: acquiring CA information for follower catalog manager: success
I20260928 16:02:45.144927 13722 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc780d0aca6b406898f0f503c7c18801: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 16:02:45.221717 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:45.216393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.22.125.253:43175)
0928 16:02:45.216636 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.216644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.216933 (+ 289us) server_negotiation.cc:407] Connection header received
0928 16:02:45.217166 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.217178 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.217249 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.217355 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:45.217852 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.218588 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.219933 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.220436 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.220574 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:45.220980 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:45.221090 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:45.221192 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:45.221361 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 16:02:45.221446 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 16:02:45.233780 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:45.227953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.22.125.252:41129)
0928 16:02:45.228197 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.228203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.228433 (+ 230us) server_negotiation.cc:407] Connection header received
0928 16:02:45.228641 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.228647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.228838 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.228967 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:45.229370 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.230112 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.231483 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.231908 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.232048 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:45.232601 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:45.233149 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:45.233319 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:45.233388 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 16:02:45.233510 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260928 16:02:45.248198 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:45.239666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.22.125.252:41129)
0928 16:02:45.239927 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.239936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.241924 (+ 1988us) server_negotiation.cc:407] Connection header received
0928 16:02:45.242182 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.242187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.242272 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.242371 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:45.243081 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.244101 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.246410 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.246926 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.247066 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:45.247472 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:45.247586 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:45.247674 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:45.247828 (+ 154us) server_negotiation.cc:299] Negotiation successful
0928 16:02:45.247904 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 16:02:45.248189 13881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:45.239361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.22.125.254:37403)
0928 16:02:45.241664 (+ 2303us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.241671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.241691 (+ 20us) server_negotiation.cc:407] Connection header received
0928 16:02:45.241826 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.241833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.242075 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.242263 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:45.242803 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.243907 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.245582 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.246022 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.246188 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:45.246731 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:45.247172 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:45.247293 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:45.247746 (+ 453us) server_negotiation.cc:299] Negotiation successful
0928 16:02:45.247852 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2170,"thread_start_us":2025,"threads_started":1}
I20260928 16:02:45.248713 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 16:02:45.240456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.22.125.253:43175)
0928 16:02:45.240683 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 16:02:45.240691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 16:02:45.242250 (+ 1559us) server_negotiation.cc:407] Connection header received
0928 16:02:45.242547 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 16:02:45.242555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 16:02:45.242648 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 16:02:45.242715 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 16:02:45.243445 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.244273 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.246557 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 16:02:45.247198 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 16:02:45.247324 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 16:02:45.247913 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 16:02:45.248064 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 16:02:45.248126 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 16:02:45.248380 (+ 254us) server_negotiation.cc:299] Negotiation successful
0928 16:02:45.248463 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 16:02:45.252655 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 13488
I20260928 16:02:45.253679 13550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:45.363970 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 13488
I20260928 16:02:45.384992 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 13563
I20260928 16:02:45.386157 13625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:45.493518 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 13563
I20260928 16:02:45.514125 23031 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu pid 13637
I20260928 16:02:45.515277 13699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 16:02:45.622169 23031 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYsP_Ep/build/asan/bin/kudu with pid 13637
2026-09-28T16:02:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4368 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8700 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (359617 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST